Azeez's Notes

6 years at WSO2!

2011-12-01T19:12:00.001+05:30

3 years ago, I wrote a blog post titled "3 years at WSO2". So, it's only befitting that I write a similar post at the end of 3 more years. These 3 years passed by very quickly. Time does fly when you are enjoying yourself.

Over the past 3 years, the company has grown from strength to strength while continuing the maintain its culture & values, and emerged as a leader in the middleware space. We've built a comprehensive middleware stack, as well as built the first complete Cloud middleware platform.

Even after 6 long years, I'm still having a lot of fun & always learning something new all the time, which keeps me excited & motivated.

Just like I did in my "3 years at WSO2" post, let me end this post with another team photo we took at ApacheCon 2011, Vancouver, Canada.

The birth of Stratos multi-tenancy

2011-11-19T06:58:00.001+05:30

On Nov 14th, 2009, I was all by myself & had a LONG & BORING 12hr transit at the Hong Kong airport! I was returning to Sri Lanka after a trip to the US, where I was a speaker at ApacheCon NA, 2009.

Since I had nothing better to do, I started looking at how multi-tenant service deployment & dispatching can be made to work in WSO2 WSAS (since then, this product has been renamed to WSO2 AppServer). I tweeted about what I was planning to do.

I started implementing this, and got very excited when the first request got dispatched to a tenant. So, again I tweeted about this (now historic) event.

This was the birth of the WSO2 Stratos Cloud middleware platform! The code I wrote during this transit formed the basis for multi-tenancy in Stratos.

During the next month, I got some of the other core stuff working, and like any good Tweep, tweeted about the progress.

My tweets had gotten Sanjiva excited, prompting him to send me the following mail.

The very first bits of the multi-tenancy code was developed in a scratch section of the WSO2 Subversion repo before it became stable enough to be committed to the trunk.

So, it's been 2 years since the first message got dispatched to a tenant at the HKG airport. Time well spent during a long transit. Over the past few years, this code was improved by many people & Stratos has grown from strength to strength to become the only complete open source Cloud middeware platform. StratoLive, the PaaS based on Stratos, went onto to become the first & only complete PaaS.

P.S. I used SnapBird to search through my old tweets.

How to customize WSO2 Carbon UI theme

2011-11-15T00:28:00.001+05:30

Chanaka has written an article showing how easy it is to introduce your own custom theme for any WSO2 Carbon based products. The interesting thing is, the theme itself is deployed as an OSGi bundle, so it is just a matter of replacing the OSGi bundle & refreshing it.

See 'Playing Around With Carbon Product Themes' for more details.

ApacheCon 2011: A multi-tenant architecture for Apache Axis2

2011-11-11T06:21:00.001+05:30

A multi-tenant architecture for Apache Axis2

View more presentations from afkham_azeez

ApacheCon 2011: Building a scalable multi-tenanted Cloud-native Application Server using Tomcat, Axis2 & Synapse

2011-11-10T08:52:00.001+05:30

Building a multi-tenanted Cloud-native AppServer

View more presentations from afkham_azeez.

Listen to the presentation audio at http://lanyrd.com/2011/apachecon-north-america/skdpt/

Lazy Loading Deployment Artifacts in a PaaS Deployment

2011-11-06T21:42:00.002+05:30

1.0 Introduction

One of the fundamental goals of Cloud computing is optimal usage of limited resources. This involves maximizing ROI for all parties concerned, and making sure that all parties win. Multi-tenancy is a core attribute of a Cloud deployment. We have to ensure that all available resources can be optimally utilized by many tenants as possible. Lazy loading plays a major role in resource usage optimization.

In this article, we will take a look at how deployment artifacts (webapps, service artifacts etc) can be optimally loaded in a Platform-as-a-Service (PaaS) implementation. We will describe how we have implemented lazy loading in the WSO2 Stratos Cloud Middleware Platform & the WSO2 StratosLive PaaS.

2.0 Lazy Loading

Lazy loading is a widely used & well-known concept in the field of computing. There are four different variants of the lazy loading pattern, as documented by Martin Fowler. The differences between these variants are subtle.

1. Lazy Initialization

A null field indicates that there is no data. When the value is requested, a null check is performed to see if the actual data needs to be loaded.

2. Virtual Proxy

The virtual proxy implements the same interface as the real object, and when called for the very first time, it loads the real object & delegates to that object.

3. Value Holder

A value holder is an object with a getValue method, which the clients will invoke in order to obtain a reference to the real object. Note that the method may not necessarily be named getValue.

4. Ghost

The real object without any data. The data is loaded as and when required.

In the next sections of the article, we will explain how we used the Value Holder & Ghost variants of the lazy loading pattern in the Stratos Cloud Middleware Platform & the StratosLive PaaS.

3.0 Lazy Loading Tenant AxisConfiguration

The Stratos deployment model revolves around the Axis2 deployment model. We have found the Axis2 deployment model to be a very flexible & powerful one, hence we decided to go ahead with the Axis2 deployment model for deploying artifacts in the WSO2 Carbon middleware platform & WSO2 Stratos Cloud middleware platform. The Axis2 deployment model totally isolates artifact deployment & classloader isolation, hence is a very good model to adopt for a multi-tenanted Cloud deployment.

When the WSO2 Stratos processes start up, we do not load all the tenants into memory. Obviously, it will not scale if we were to do so. Tenants are loaded only when they are needed, and if a tenant is not being used for a certain time period, we unload such tenants. Tenants will be loaded in the following scenarios;

1. A request is received for an artifact deployed by the tenant.

These can be requests to webapps, Web services, proxy services, tenant UI components and so on.

2. A tenant's management console view is loaded.

Tenant's can manage services in their view of the PaaS using the management console. When tenants want to perform such management activities, we load the tenant.

3. A periodic task scheduled for the tenant gets activated

In a PaaS, tenants should be able to schedule tasks that run periodically. In such cases, when the task starts to run, if the tenant has not been loaded, we will load the tenant.

In WSO2 Stratos, loading a tenant mainly translates to creation of an Axis2 Configuration & ConfigurationContext in programming terms. When an AxisConfiguration is created for a tenant, a series of OSGi service calls results in the relevant permission scheme & other metadata being loaded. These are lightweight service calls that do not impose high overhead.

We have used the ValueHolder variant of lazy loading in this instance. The relevant method call is;

TenantAxisUtils.getTenantConfigurationContext(String tenantDomain)

The getTenantConfigurationContext method is the equivalent of the getValue method in the ValueHolder. This method will create & return a new Axis2 ConfigurationContext for that tenant domain, if it does not already exist, and will return the already created ConfigurationContext if the tenant has already been loaded.

One shortcoming of loading the entire AxisConfiguration is that all artifacts belonging to that particular tenant will also be loaded. Hence, the time taken for the execution of this method will increase with the number of artifacts deployed by the tenant. This can lead to unfavorable consequences such as the very first request received for a tenant always timing out. Such requests are targeted at a single artifact, but loading the AxisConfiguration means the entire Axis2 repo gets loaded, hence we will end up loading all artifacts, leading to unnecessary memory usage & processing time. In section 4.0, we will describe how we can overcome this issue by using the Ghost variant of the lazy loading pattern.

4.0 Lazy Loading Tenant Deployment Artifacts

As described in section 3.0 above, loading an entire tenant AxisConfiguration can lead to increasing first-request response times since all artifacts have to be loaded. We can overcome this by loading the required artifact into a tenant's AxisConfiguration on demand.

We use the Ghost variant of the lazy loading pattern to accomplish this. Let's take a look at how normal artifact deployment works.

Figure 1: Deployment without Ghost Deployer & Ghost Dispatcher

As shown in the above diagram, a set of Axis2 deployers are registered for each artifact type. The DeploymentEngine maintains a collection of such deployers, and then when an AxisConfiguration is created, it will call all the deployers to load all the artifacts found in the Axis2 repository. The deployers will load all the artifacts into memory. Needless to say, this will take a long time if there are many artifacts to be loaded. Next let us look into the concept of Ghost artifact deployment, to see how we reduce this initial loading time.

Figure 2: Ghost Deployment with Ghost Deployer & Ghost Dispatcher

As shown in the above diagram, we change the deployment slightly & register GhostDeployers for all artifact types. The GhostDeployer instance will maintain references to the actual Axis2 deployers. Let us look at the sequence of events.

1. When a new artifact is deployed, either by uploading using the Management console UI, or copying the artifact to the Axis2 repo, the GhostDeployer will call the real deployer, which will create the real Axis2 Service

2. The GhostDeployer will then pick up that Axis2 service, and create a simple metadata file which has some basic information needed to create a Ghost Axis2Service. This will mainly contain the service name & operations; information needed for dispatching requests.

3. On subsequent deployments of the service which was first created in step 2, the GhostDeployer will pick it up from the metadata file created in step 2 above.

4. Next the GhostDeployer will create a Ghost Axis2 Service which will contain a special parameter (Ghost Parameter) which will identify the service as a Ghost service, and register that service into the AxisConfig

5. Next, say a request comes in for the said service, which at the moment is deployed in Ghost form.

6. There is a special Axis2 dispatching handler called the GhostDispatcher, which is registered after all standard Axis2 dispatchers. It will check the Ghost Parameter added to the service in step 4 above to check whether the Actual service deployment needs to take place, if the service in concern is a Ghost service.

7. If the service is determined to be a Ghost service in step 6 above, the GhostDispatcher will call the GhostDeployer to carry out the actual service deployment. At that point, the GhostDeployer will call the actual deployer, which will load the rest of the metadata & information required by the actual service. This will load all metadata from the registry, load service policies & so on.

In addition, if certain services are not used for some time, they will be undeployed & reloaded in Ghost form. This will ensure that rarely used services, or services that don't get called too often, do not unnecessarily use up resources.

5.0 Performance Comparison

We carried out performance comparisons & plotted the graph of "Initial Response Time" vs. "Number of Artifacts". The initial response time is the response time seen by a client when a tenant has not been loaded. The number of artifacts is the number of deployment artifacts deployed under that tenant.

We sent requests targeted at a particular operation in a particular service. We recorded the initial response times while linearly increasing the number of deployment artifacts up to 300 in that tenant. The resulting graph is shown below.

Figure 3: Performance comparison - with & without Ghost Deployment

As can be seen, without Ghost Deployment, the initial response time linearly increases. Eventually, clients sending the very first requests will start timing out. With Ghost Deployment, we ensure that the first response time is linear, hence this will improve the client experience on first request, and will also ensure that the first requests don't timeout.

Ghost Deployment is one of the major enhancements we introduced in the recent StratosLive upgrades to enable our tenants to experience better performance.

6.0 Conclusion
Lazy loading is a core requirement in Cloud deployments. In this article we looked at some of the core concepts behind lazy loading. We saw how Ghost Deployment yields drastic performance improvements in a multi-tenanted PaaS environment such as StratosLive. Future enhancements in this area in Stratos will include lazy loading of webapps using a GhostWebappDeployer Tomcat valve.

WSO2Con 2011: Panel: Cloud and SOA: The good, the bad and the ugly

2011-10-31T17:11:00.001+05:30

WSO2Con 2011: Building Cool Applications with WSO2 StratosLive

2011-10-31T16:51:00.001+05:30

WSO2Con 2011: Open Source Middleware for the Cloud

2011-10-31T16:41:00.001+05:30

My Top 10 Third Party IntelliJ IDEA Plugins

2011-10-28T11:16:00.001+05:30

iPaid

2011-10-20T19:48:00.001+05:30

Colombo, Sri Lanka

2011-10-18T17:09:00.001+05:30

Colombo

View more presentations from afkham_azeez.

ApacheCon NA 2011, Vancouver

2011-10-14T10:28:00.000+05:30

I will be conducting a couple of sessions at ApacheCon NA 2011, which will be held in the beautiful seaport city of Vancouver.

Building a scalable multi-tenant Application Server on the Cloud using Tomcat, Axis2 & Synapse

November 9 5:00PM

Apache Tomcat is one of most popular & widely used Application Servers, and Apache Axis2 is one of most widely used Java Web services servers. Apache Synapse is one of the popular, high performant ESBs widely used in the industry.

In this session, we will look at how we combine these great projects from the ASF into building a scalable, elastic,multi-tenanted Application Server, which allows you to deploy cloud-native webapps on the Cloud, and benefit from all the advantages that Cloud Computing brings in. We will also see how easy it is to deploy any standard webapp on the Cloud, and seamlessly integrate with the authentication, authorization & management infrastructure provided by the underlying Platform-as-a-Service (PaaS).

This session will be co-presented by Selvaratnam Uthaiyashankar, Senior Manager & Senior Architect at WSO2, and Apache Member.

An architecture for enabling multi-tenancy for Apache Axis2

November 10 11:30AM

Web services is the most popular technology for implementing an SOA. SOA & Cloud computing are a natural fit. Apache Axis2 is the most widely used Java Web services server in the industry. Deploying Web services on the Cloud would give all the benefits provided by Cloud Computing. However, deploying Web service applications on the Cloud requires that sharing of resources & infrastructure has to be optimally done without compromising security.

In this session, we will look at an architecture which will allow you to deploy Web services within a single Axis2 cluster in a multi-tenant fashion. We will also look at how to adopt this architecture for deploying Axis2 Web services as SaaS. We will demonstrate this in practice using the WSO2 Stratos Platform-as-a-Service (PaaS)

This session will be co-presented by Senaka Fernando, Associate Technical Lead & Product Manager - WSO2 Governance Registry, at WSO2, and Apache Member.

In these sessions, we will be talking about the practical realities of building Cloud Middleware Platforms based on our real world experiences, so don't miss it!

How to setup WSO2 Elastic Load Balancer

2011-09-24T21:14:00.001+05:30

In "WSO2 Load Balancer - how it works" we discussed the internal workings of the WSO2 Elastic Load Balancer. In this article, we will discuss in detail how to configure this load balancer. Even though not necessary, understanding how the WSO2 load balancer works will enable the reader to understand the rationale behind the way certain things are configured.

1.0 Example Setup

Let us take the example of configuring the following setup shown in the diagram below. We will look at how to configure the elastic load balancer as well as the worker node clusters in this setup. By understanding this setup, you should be able to setup WSO2 ELB fronted clusters with any of the WSO2 Carbon based products or WSO2 Stratos Services.

In the above setup, we will have a LB cluster setup in primary-secondary configuration. This LB cluster will be fronting 2 clusters; App Server cluster & Data Services Server cluster.

In this exercise, we will assume the following ports.

Load Balancer
HTTP port: 80
HTTPS port: 443

App Server
HTTP port: 9762 --> proxied via port 80
HTTPS port: 9443 --> proxied via port 443

Data Services Server
HTTP port: 9762 --> proxied via port 80
HTTPS port: 9443 --> proxied via port 443

We will also assume well-known address based membership discovery (wka) to keep the article simple. The same configuration will work with multicast based membership discovery too, with the membershipScheme set to multicast in the axis2.xml file.

In the following sections, we will look at how to configure the worker node clusters & the ELB.

2.0 Configuring the App Server Cluster

You will have to change the mgt-transports.xml, carbon.xml & axis2.xml file in order to configure the App Server cluster. These files are available in the CARBON_HOME/repository/conf directory.

2.1 mgt-transports.xml

Uncomment the HTTP proxy port in this file. The HTTP proxy port has to be set to port 80. In this case, the LB is the HTTP proxy, and we are informing the App Server, that it is fronted by a proxy where the HTTP port is 80.

 
<transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
    <parameter name="proxyPort">80</parameter>
</transport >

Uncomment the HTTPS proxy port in this file. The HTTPS proxy port has to be set to port 443. In this case, the LB is the HTTPS proxy, and we are informing the App Server, that it is fronted by a proxy where the HTTPS port is 443.

 
 <transport name="http" class="org.wso2.carbon.server.transports.http.HttpTransport">
    <parameter name="proxyPort">443</parameter>
 </transport >

2.2 carbon.xml

In this file, we have to specify the Host of the App Server. In this example, it is appserver.stratoslive.wso2.com. This is configured by uncommenting the HostName entry in the carbon.xml file and specifying the host as shown below.

<HostName>appserver.stratoslive.wso2.com</HostName>

2.3 axis2.xml

In the axis2.xml file, you have to enable the clustering. This is used for mainly membership discovery. As explained in "WSO2 Load Balancer - how it works", the LB discovers worker nodes using the underlying Axis2 clustering mechanism.

      
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
    <parameter name="membershipScheme">wka</parameter>
    <parameter name="localMemberHost">appserver.private.stratoslive.wso2.com</parameter> 
    <parameter name="localMemberPort">4100</parameter>
    <parameter name="domain">wso2.as.domain</parameter>
    <members>
        <member>
            <hostName>appserver.stratoslive.wso2.com</hostName>
            <port>4000</port>
        </member>
    </members>
</clustering>

Note that appserver.stratoslive.wso2.com maps to the public IP address of the load balancer. The localMemberHost can be an IP address which has private visibility. You can keep the localMemberHost entry empty, and if it is empty, one of the available interfaces of the machine the App Server node is running on will be picked. The important thing to note here is that the clustering domain has been set to wso2.as.domain.

3.0 Configuring the Data Services Server Cluster

You will have to change the mgt-transports.xml, carbon.xml & axis2.xml file in order to configure the Data Services Server cluster, and the configuration is almost the same as that of the App Server cluster.

3.1 mgt-transports.xml

The configuration is identical to what we saw in section 2.1 in the App Server cluster configuration section.

3.2 carbon.xml

In this file, we have to specify the Host of the Data Services Server. In this example, it is data.stratoslive.wso2.com. This is configured by uncommenting the HostName entry in the carbon.xml file and specifying the host as shown below.

<HostName>data.stratoslive.wso2.com</HostName>

3.3 axis2.xml

Most of the configuration is identical to what we've seen in section 2.3. The main difference will be that the clustering domain has to be set to wso2.ds.domain. The relevant configuration is shown below.

<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
    <parameter name="membershipScheme">wka</parameter>
    <parameter name="localMemberHost">data.private.stratoslive.wso2.com</parameter> 
    <parameter name="localMemberPort">4100</parameter>
    <parameter name="domain">wso2.ds.domain</parameter>
    <members>
        <member>
            <hostName>data.stratoslive.wso2.com</hostName>
            <port>4000</port>
        </member>
    </members>
</clustering>

Note that appserver.stratoslive.wso2.com maps to the public IP address of the load balancer.

4.0 Configuring the Elastic Load Balancer Cluster

axis2.xml

In this file, locate the transportReceiver entries for the HTTP & HTTPS ports and change them to port 80 & 443 respectively. The relevant XML segments are shown below.

<transportReceiver name="http" class="org.apache.synapse.transport.nhttp.HttpCoreNIOListener">
    <parameter name="port" locked="false">80</parameter>
</transportReceiver>

<transportReceiver name="https" class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener">
    <parameter name="port" locked="false">443</parameter>
</transportReceiver>

loadbalancer.xml

Generally, you will only need to change the CARBON_HOME/conf/loadbalancer.xml file. All other relevant files have the proper defaults, and will have to be changed only in the case of advanced configurations.

The following segment in the loadbalancer.xml file shows how we map the HTTP host to the clustering domain.

<loadBalancerConfig xmlns="http://ws.apache.org/ns/synapse">
    <services>
        <service>
            <hosts>
                <host>appserver.stratoslive.wso2.com</host>
            </hosts>
            <domain>wso2.as.domain</domain>
        </service>
        <service>
            <hosts>
                <host>data.stratoslive.wso2.com</host>
            </hosts>
            <domain>wso2.ds.domain</domain>
        </service>
    <services>
</loadBalancerConfig>

4.1 Disabling Autoscaling

At the time of writing, the WSO2 Elastic Load Balancer only supports autoscaling on Amazon EC2 & compatible IaaSs since it is based on the EC2 API. So, you will see error messages if you try to run this setup on a local network, for example. So, in such cases, you will have to disable autoscaling. You have to do the following to disable autoscaling;

Delete $CARBON_HOME/repository/deployment/server/synapse-configs/tasks/autoscaler.xml
Remove the autoscaleIn & autoscaleOut mediators from $CARBON_HOME/repository/deployment/server/synapse-configs/sequences/main.xml

5.0 Testing the Setup

Now we have completed setting up the system, so it is just a matter of starting up the nodes. If you have autoscaling setup, you will simply have to start only the Elastic Load Balancer. This LB will take care of starting up all other nodes in the system. If everything is properly setup, you should see messages on the LB log file saying members from wso2.as.domain & wso2.ds.domain joined the cluster.

To make sure that everything is working properly, in your client machine, map the appserver.stratoslive.wso2.com & data.stratoslive.wso2.com names to the relevant IP address of the primary Elastic load balancer & go to the following URLs using your Web browser.

If you have properly setup your cluster, you should see the login screens of the relevant WSO2 product.

IntelliJ IDEA - The best Java IDE!

2011-09-22T10:50:00.001+05:30

Intelli J IDEA

View more presentations from afkham_azeez

Sun Certified Motor Mechanic (SCMM)

2011-09-14T21:17:00.000+05:30

Today just before my session on WSO2 Stratos at WSO2Con, I got to know that I have a qualification which I didn't even know existed. The compere, Geethika Cooray, introduced me as a Sun Certified Motor Mechanic! I didn't even know Sun was providing such a certification :) Anyway folks, now it is official. Now I'm a Sun Certified Motor Mechanic :D

WSO2Con 2011: Introduction to WSO2 Stratos

2011-09-14T16:30:00.000+05:30

WSO2con 2011: Introduction to Stratos

View more presentations from afkham_azeez

WSO2Con 2011: Introduction to the WSO2 Carbon Platform

2011-09-14T14:03:00.002+05:30

WSO2Con 2011: Introduction to the WSO2 Carbon Platform

View more presentations from afkham_azeez

This blog template uses SyntaxHighlighter hosted on StratosLive

2011-09-10T00:51:00.001+05:30

I have included SyntaxHighlighter hosted on WSO2 StratosLive under my tenant azeez.org in this blog's template as shown in the above figure. If you'd like to include syntax highlighting on your blog & want to use this hosted SyntaxHighlighter, you need to add the following in the head section of your template:

             
<link href=" http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Styles/SyntaxHighlighter.css" rel="stylesheet" type="text/css"></link>
<script src=" http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shCore.js" type="text/javascript"/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushCpp.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushCSharp.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushCss.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushJava.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushJScript.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushPhp.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushPython.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushRuby.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushSql.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushVb.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushXml.js' type='text/javascript'/>
<script src=' http://appserver.stratoslive.wso2.com/t/azeez.org/webapps/highlighter/Scripts/shBrushPerl.js' type='text/javascript'/>

<script language='javascript'>
    window.onload = function () {
        dp.SyntaxHighlighter.ClipboardSwf = '/Scripts/clipboard.swf';
        dp.SyntaxHighlighter.HighlightAll('code');
     }
</script>

Now, whenever you need to include highlighted code, you simply have to do the following;

<pre name="code" class="c-sharp">
... some code here ...
</pre>

For more information, see http://code.google.com/p/syntaxhighlighter/wiki/Usage

Embedding Tomcat 7

2011-09-09T22:09:00.001+05:30

The following code shows how easy it is to embed Apache Tomcat 7 using the API provided by org.wso2.carbon.tomcat.BetterTomcat. BetterTomcat is a class which wraps an org.apache.catalina.startup.Tomcat object & provides a simpler API for some common scenarios. This class provides a simple API which enables you to embed & start Tomcat by writing just a few lines of code as shown below.

public void startServer() throws ServletException, LifecycleException { 
    BetterTomcat tomcat = new BetterTomcat(8080); 
    tomcat.addWebapp("/hello", WEBAPP_PATH); 
    tomcat.start(); 
}

The following code shows how to start an embedded Tomcat server with one HTTP connector on port 8080 & one HTTPS connector on port 8443.

public void startMultipleConnectors() throws LifecycleException, ServletException { 
    BetterTomcat tomcat = new BetterTomcat(); 
    tomcat.addConnector(BetterTomcat.Protocol.HTTP_11_NIO, 8080); 
    Connector https = tomcat.addConnector(BetterTomcat.Protocol.HTTPS_11_NIO, 8443); 
    https.setProperty("keystoreFile", KEYSTORE_PATH); 
    https.setProperty("keystorePass", "wso2carbon"); 
    tomcat.addWebapp("/hello", WEBAPP_PATH); 
    tomcat.start(); 
}

The BetterTomcat class is used for starting up embedded Tomcat 7, which forms the core of WSO2 Carbon, WSO2 AS, WSO2 Stratos & StratosLive. If you are interested in checking out the code, have a look at the following links.
1. https://svn.wso2.org/repos/wso2/trunk/carbon/core/org.wso2.carbon.tomcat
2. https://svn.wso2.org/repos/wso2/trunk/carbon/core/org.wso2.carbon.tomcat/src/main/java/org/wso2/carbon/tomcat/BetterTomcat.java

The org.wso2.carbon.tomcat OSGi bundle (#2 above) is a self contained jar file which contains all the dependencies needed to run embedded Tomcat.

Please note that this is not distributed as a separate product, but only a class in the WSO2 Carbon core libraries, and is only used internally.

Tomcat is a trademark of the Apache Software Foundation.

[WSO2Con 2011] Introduction to the WSO2 Carbon Platform at WSOCon 2011

2011-09-09T19:07:00.001+05:30

Introduction to the WSO2 Carbon Platform is one of tutorials which will be conducted by Supun Malinga & me at the WSO2Con next week. This is a 3-4 hour session where we will be covering all the core concepts related to the award-winning WSO2 Carbon platform. Most importantly, you will be able to get an good idea about the entire suite of WSO2 products, where each of these product fit-in in an SOA deployment & what products/components should be used where in your Solution Architecture.

In addition to getting a complete understanding about this platform, you will also learn the finer details about the platform, and the innovative aspects introduced by the Carbon framework. We will be having a very interesting hands on session where the participants will learn how to write a Carbon component & integrate it with a Carbon server instance. You will also be able to learn the development lifecycle of a Carbon component.

I'd recommend you to first attend the OSGi training session conducted by Sameera & Pradeep, since that will enable you to better understand some of the Carbon platform concepts. The Carbon platform is based on OSGi.

Is PaaS on top of IaaS the proper way to go?

2011-09-08T08:54:00.003+05:30

Traditionally, when people talk about Cloud computing, people talk about SaaS, PaaS & IaaS as depicted in the following diagram.

So, traditionally people think about building a PaaS on top of an IaaS. Samisa recently blogged about the practical problems we have been facing with IaaS, in particular, Amazon Web Services (AWS). To be honest, working with AWS has been a big challenge since we experienced very bad performance when it comes to IO, as well as the infrastructure itself behaving in a very unpredictable manner from time to time. Every so often, we would lose the network, and we would not even be able to connect to 127.0.0.1 from our software, forcing restarts. So much for SLAs & high availability! It is also well known that virtualization leads to IO performance degradation, so if your PaaS or SaaS is IO intensive, you may see a considerable drop in performance.

Coming to think about it, running a PaaS on top of an IaaS such as AWS could be overkill. In such a setup, we have two levels of multi-tenancy; one at the IaaS layer, where the PaaS service provider is a tenant, the other level is at the PaaS layer itself. One level of multi-tenancy at the PaaS level is what is actually required. When it comes to elasticity in a PaaS, what we actually need is a new process; in the case of Java PaaS, a new JVM, but what we do in a setup such as the one shown in the above figure is, we spin up a new image instance (in the case of AWS, a new EC2 instance), and then start a process in that new instance. Spinning up a new instance can take up to 15 minutes, so by the time a new instance boots up & is able to perform some work, the need for starting up that instance may have passed due to the traffic dropping back to normal levels.

Perhaps, the proper model would be to run your PaaS on the infrastructure (hardware+networking+OS) directly without virtualization as shown in the above diagram, and have a few cold standby EC2 instances for Cloud bursting. This is the model we will have to go with, at least until the time where we have IaaSs that are much more stable. Another advantage is, the cost of having your own hardware will be very much less than the accumulated amount you would be paying the IaaS provider, since you will be running your PaaS 24x7.

The challenges of going for such a setup include, having to implement alternatives to many of the functionality that is already provided by the IaaS. This includes geographically distributed deployments (AWS provides this through availability zones & regions), firewall functionality (AWS provide this using security groups), public IP address assignment (AWS provides this through Elastic IPs), and so on. However, the benefits of implementing such functionality at the infrastructure level will yield huge benefits for large scale Platforms-as-a-Service such as StratosLive.

WSO2 Load Balancer - how it works

2011-09-07T20:15:00.002+05:30

... an under the hood look at the WSO2 Elastic Load Balancer.

In The role of a Load Balancer in a Platform-as-a-Service, I explained some key concepts behind load balancing. In this article, we will take a look at the inner workings of the WSO2 Elastic Load Balancer to understand how some of these concepts have been incorporated. The WSO2 Elastic Load Balancer is the load balancer used in StratosLive, the first fully complete open Java Platform-as-a-Service (PaaS)

High Level Architecture

As shown in the diagram above, the WSO2 Elastic Load Balancer is built using Apache Tribes, which is the group management framework used by Apache Tomcat as well as Apache Axis2, Apache Axis2 clustering module, Apache Synapse - one of the best performant mediation frameworks, and the autoscaling component from the award-winning WSO2 Carbon framework. The autoscaling component interacts with the Amazon EC2 API to carry out infrastructure related functionalities such as starting new instances, terminating running instances, mapping elastic IPs & so on.

Let's start looking at the above diagram from the topmost layer. The Carbon autoscaling component is responsible for keeping track of the traffic to each Cloud Service cluster, and making decisions related to scaling the system up when the load increases & scaling the system down when the load reduces. It is also responsible for keeping a sanity check on the system. This sanity check ensures that the minimum system configuration, such as the minimum number of running instances of each Service & load balancer cluster is maintained at all times. This autoscaling component calls out to the EC2 API for infrastructure related activities. The autoscaling component has been implemented as two Apache Synapse mediators; autoscale-in & autoscale-out mediators, and a Synapse task.

A Synapse endpoint called the ServiceDynamicLoadBalanceEndpoint is responsible for routing the messages to the appropriate node. First, the Cloud Service cluster to which the message has to be sent is identified using the Host HTTP header. Then a member from that Service cluster to which the message has to be routed to is selected according to the load balancing algorithm specified, and the message is sent to that member. The response from that member is sent back to the client which originated the request. While the LB is trying to forward a request to a member, that member may fail, and in such a case, if possible, the ServiceLoadBalanceEndpoint will try to failover to another available member. Such a member which failed will be suspended for a specified period. This endpoint is also responsible for handling sticky sessions. In fact, when a request comes in, this endpoint first checks whether there is sticky session created for that client, and if such a session is found, the request is forwarded to the relevant member. Sticky sessions are identified using the value of the JSESSIONID cookie.

Binary Relay or Message Relay is the message pass through mechanism in the WSO2 Elastic Load Balancer. This relay enables the LB to pass the messages without building or processing the message. This ensures that the overhead introduced by the LB is minimal.

Membership discovery & management is handled by the Axis2 Clustering module, which handles Service membership. There are multiple membership handlers, one for each clustering domain, responsible for handling membership. The ServiceDynamicLoadBalanceEndpoint will obtain the relevant members of the cluster from this Axis2 membership handlers, and then apply the load balance algorithm to determine the next member to which the request has to be sent. The Axis2 clustering implementation uses the underlying Apache Tribes Group Management Framework for group membership management.

Dynamic Clusters & Group Management Agents

In the WSO2 Elastic Load Balancer, we have support for dynamic & hybrid membership discovery. As shown in the above diagram, the Elastic Load Balancer can be set up in a primary-secondary configuration. These LBs will be in a cluster, and state replication can take place between these nodes. The LBs are also configured as Group Management Agents, hence they can manage domains A, B & X. This means, any membership changes in the domains, A, B, X will be visible to LBpri & LBsec. A public IP address will be assigned to the LBpri in a typical deployment. The LBsec will keep observing LBpri, and if LBpri fails, LBsec will map the public IP to itself, spawn another secondary LB, and then become the primary LB. The client sees the public IP address mapped to LBpri, and needs to send the HTTP 1.1 Host header. This Host header will be used by LBpri to determine the destination Service cluster. For example, if the Host is set as Hb, the LBpri will select one of the nodes in domain B, and send the request to it.

For this setup to work, clustering has to be enabled in all the backend worker nodes, as well as the load balancers, because the underlying Axis2 clustering mechanism & Tribes group management framework are used for dynamic membership management.

Real Port to Proxy Port Mapping

The mapping between the proxy port & the real port on the member (worker node) is provided as shown in the diagram above. In the above diagram, we have shown an example of the LB exposing 4 ports; 2 for HTTP & 2 for HTTPS requests. The HTTP ports on the LB are 80 & 8280. The HTTPS ports on the LB are 443 & 8243.

Member A1 exposes 4 ports; HTTP 9762 is proxied via port 80 in the LB, HTTP 9763 is proxied via port 8280 in the LB; HTTPS 9443 is proxied via port 443 in the LB & HTTPS 9444 is proxied via port 8243 in the LB.

Member A2 also has a similar real port to proxy port mapping as depicted in the diagram above.

These ports & the mapped proxy ports will be advertised by the members when they join the cluster, and the LB can retrieve these values as properties of the member. When the LB receives a request, it will get the incoming port, and before dispatching the request to a member, it will try to get the mapped real port from the member, and route the request to the appropriately mapped port (as advertised by the member).

The diagram shown below shows how requests from clients will be proxied from the LB to member A1.

How Elasticity/Autoscaling is Handled

Autoscaling is handled by the WSO2 Carbon autoscaling component. This component keeps track of the number of messages in flight to each Service cluster, and decides whether to scale the system up or down. The autoscaler component consists of an AutoscaleIn mediator, AutoscaleOut mediator & LoadAnalyzerTask, as shown in the diagram below. When a message is received, the AutoscaleIn mediator creates a unique token & puts it into a list. When a response to the message is sent, the AutoscaleOut mediator removes that token from that list. So, this list tracks the number of messages in flight for each backend Service. The LoadAnalyzerTask periodically check the list lengths & based on the configuration parameters, decides whether to scale up or scale down a backend Service.

This component will start new Service member instances, and once those members successfully boot up, they will join the relevant Service cluster. Now the load balancer will start forwarding the request to the new members as well.

Membership Management using Apache Tribes

As shown in the above diagram, the WSO2 Elastic Load Balancer is a special Tribes member, which can sense membership changes in the groups it is managing. Each group has its dedicated membership channel, and the LB can connect to all these channels. The other Service members can only see membership changes in their respective channels. In the above diagram, members in Group A (A1, A2, A3,..., An) can sense membership changes in Membership Channel - A, but cannot see such changes in Membership Channel -B. However, the LB can see membership changes in Channels A, B & LB.

Deployment in a Data Center

If your network allows multicasting, you could use the multicast based membership scheme. In this case, membership will be handled using multicasting, hence a multicast socket needs to be configured.

Deployment on the Cloud

On a Cloud deployment, typically you would use well-known address based membership scheme since multicasting will be disabled on such setups. In such a setup, we will make the one or more LBs to be the well-known members. All nodes in the Service clusters, SA, SB & SX will see the LB as a well-known member. Membership will revolve around the well-known load balancer in this setup.

WSO2 Elastic Load Balancer on StratosLive

The above diagram shows how we have deployed the WSO2 Elastic Load Balancers in StratosLive. The diagram shows how a single LB instance can front several Cloud Service clusters. For example, the Manager, Business Rules Server (BRS) & Business Process Server (BPS) Cloud Services are fronted by a single load balancer, and the Governance Registry (G-Reg), Mashup Server (MS) & Gadget Server (GS) Cloud Services are fronted by another load balancer. Some other services such as the Identity Server, Application Server, Business Activity Monotor (BAM) & Data Services Server (DSS) have dedicated load balancers, since these are popular Services with heavy incoming traffic. As you may note, some nodes are marked as permanent nodes. This means, these nodes are not supposed to be shutdown if the autoscaler makes a scale-down decision. Special flags mark these nodes as permanent nodes, and the autoscaler will check this flag before deciding to terminate an instance. It is the elastic nodes that can be terminated if a scale-down decision is made.

The role of a Load Balancer in a Platform-as-a-Service

2011-09-02T21:51:00.014+05:30

Introduction
Load balancing is a key concern in a Platform-as-a-Service (PaaS) or a middleware platform deployed on the Cloud. In this article, we will look at some key concepts related to load balancing, and the the significance & usage of load balancers in PaaS deployments.

The primary functionality of a load balancer is to distribute the load of the incoming traffic amongst a set of backend worker nodes. This set of worker nodes can be either statically configured or can be dynamically discovered. Traditionally, load balancers are configured with a set of static nodes. This means, new nodes outside this set cannot be added at runtime. Dynamic load balancers support addition & removal of worker nodes at runtime, and the IP addresses & other connection details of the backend nodes need not be known in advance by the load balancer. The load balancing algorithm is a central part of a load balancer. This algorithm specifies the load balancing policy, or how the load has to be distributed across multiple backend worker nodes. Generally, all worker nodes have identical hardware & software configuration, as well as host identical copies of deployment artifacts. Hence the round-robin load balancing algorithm is very suitable & widely used for such deployments.

Session Affinity

Most modern load balancers support session affinity. This means, if the client sends a session ID, the load balancer will forward all requests containing a particular session ID, to the same backend worker node, irrespective of the specified load balancing algorithm. This may look like defeating the purpose of load balancing. But before the session is created, the request will first be dispatched to the worker node which is next in-line, and a session will be established with that worker node. We also have to keep in mind that stateful applications inherently do not scale well, and state replication can have huge overheads, so it is best to minimize server side state if you want your application to be massively scalable. So, session-affinity based load balancing is a compromise solution to the problem of deploying stateful applications in clusters.

Elastic Load Balancer

An Elastic Load Balancer (ELB), in addition to carrying out its functionality in load balancing, is also responsible for monitoring the load & starting up new worker nodes or terminating existing worker nodes, depending on the load. This behavior of scaling up the system while the load increases & scaling down the system when the load decreases is known as autoscaling.

In a typical architecture, load balancing & autoscaling will be handled by two logically distinct components. It may even be possible to deploy the load balancer component & the autoscaler component separately.

Cloud-nativity & Load Balancing

Load balancing is the key to Cloud-based deployment architectures. The Elastic Load Balancer is an essential component in the deployment architecture, when it comes to realizing the Cloud native attributes of multi-tenancy, elasticity, distributed & dynamic wiring, and incremental deployment & testability.

Fronting Multiple Clusters - Service-aware Load Balancing

When it comes to production deployments, a load balancer does not do much of the real work. The real work is done in the backend worker nodes. So, having load balancers introduces additional cost. Since load balancers do not do much of the real work, typically, the load on the load balancer itself should be very small. Hence, a load balancer is generally capable of fronting quite a large number of backend worker nodes. In a traditional deployment, one LB may front a cluster of homogenous worker nodes. However, a load balancer is generally capable of handling multiple clusters as shown in the figure above. The important thing to note is, the traffic has to be routed to the correct cluster, and the load has to be balanced according to the specified load balancing algorithm specified for that cluster. A cluster of homogeneous worker nodes is called a Cloud Service, in Cloud deployments. So, a load balancer which fronts multiple Cloud Services is typically called a Service-aware load balancer.

Multi-tenancy - Tenant-aware Load Balancing

If a Cloud deployment has to be able to scale to thousands, hundreds of thousands or millions of tenants, we need tenant-partitioning. This means, for a single Cloud service, there will be multiple clusters, and each of these Service clusters will handle a subset of the tenants in the system. Creating dynamic tenant clusters & tenant partitioning strategies are some of the ongoing research areas. In such a tenant-partitioned deployment, the load balancers themselves need to be tenant-aware, in order to be able to route the requests to the proper tenant clusters. This means the load balancer has to be tenant-aware as well as Service-aware, since it is the Service clusters that are partitioned according to the tenants.

Single Point of Failure?

The load balancer itself can become a single point of failure, defeating the purpose of having clustered deployments. This can be handled by having the LBs deployed in pairs in either hot-hot or hot-cold configuration. If the LBs are deployed in hot-hot configuration, we could do DNS round-robin between these LBs. In hot-cold setups, if the primary LB fails, we could do an IP remapping.

WSO2 Elastic Load Balancer

The WSO2 Load Balancer (LB) is a load balancer based on Apache Synapse & WSO2 Carbon. This is also an Elastic Load Balancer, and has been deployed on StratosLive, the Platform-as-a-Service from WSO2. It is also available for download as part of the WSO2 Stratos Cloud Middleware Platform. At present, the WSO2 Elastic Load Balancer (ELB) is only Service-aware, and can be deployed as a load balancer cluster.

Further Reading

[WSO2Con 2011] Open Source Middleware for the Cloud: WSO2 Stratos

2011-08-30T18:39:00.000+05:30

I will be presenting this year as well at WSO2Con about the WSO2 Stratos Cloud Middleware Platform as well as WSO2 StratosLive Platform-as-a-Service.

Here is the abstract of my talk.

"More enterprises are recognizing the opportunity to extend the reach and cost-efficiency of their applications by delivering them as software-as-a-service (SaaS). However, the approach to deploying in the cloud and the choice of either cloud middleware software or a platform-as-a-service (PaaS) can significantly affect the success of a SaaS implementation.

WSO2 Stratos is a complete enterprise-ready cloud middleware platform designed to extend SOAs to the cloud, and it is the software that powers the WSO2 StratosLive PaaS. By providing WSO2 Carbon products as services over public, private, and hybrid cloud infrastructure, WSO2 Stratos offers an ideal platform for SaaS developers to create, manage and run enterprise-class applications and services with all the inherent benefits of a true cloud-native environment.In this session, we will be looking at the WSO2 Stratos cloud middleware platform and the benefits it offers in developing, testing, deploying and managing cloud-native applications."