Stefan's SOA & Enterprise Architecture Blog

New German Article about Enterprise Architecture Management and Cloud Computing

2011-04-14T01:36:00.001-07:00

Today we published a new article about the role of Enterprise Architecture in Cloud Computing. (This is german-only)

More Information in German:

Heute ist ein neues Online-Special des Objektspektrum erschienen mit dem Themenschwerpunkt Enterprise Architecture Management. Mit enthalten ist ein Artikel von mir und meinen Kollegen aus dem Oracle Architekturboard, Nicole Holthöfer und Berthold Maier, mit dem Titel “Cloud Enterprise-Architekturen: Die Rolle von EA für Cloud Computing-Infrastrukturen”.

Siehe Link zum Online-Special

Direkter Download des Artikels als PDF hier.

DOAG 2010: German User's Group Conference Program Online!

2010-08-17T11:38:00.001-07:00

The program of this years DOAG 2010 Conference from Nov 16 to 19,2010 in Nuremberg, Germany is online!
See http://www.doag.org/konferenz/doag/2010/programm (currently German only)
Please don't miss my presentation on day 2 (Nov 17th) about "Oracle BPEL PM - Performance Tuning and Clustering Best Practises".

See you in Nuremberg!

Announcement: Next DOAG Regional Meeting 8.9.2010 covers Enterprise Architecture

2010-08-17T00:59:00.001-07:00

The next regional meeting of DOAG in Berlin happens on September 8th in the Oracle Office in Berlin-Tegel. See http://www.doag.org/regio/berlin/i_text for full details.

18:15 - 19:15 Topic 1: Enterprise Architecture
Presenter: Thomas Baumgart, Enterprise Architect / Oracle Germany

19:30 - 20:15 Topic 2: ORACLE Enterprise Content Management
Presenter: Tino Albrecht, Oracle Germany

Web Service Transactions Part 3: Testing Rollback between multiple SOA composites

2010-08-16T03:03:00.001-07:00

Overview

The previous sample (Part 2) only contained a call to one web service from a SOA composite. It really gets interesting only when multiple distributed web services join one atomic transaction. To be able to execute this including rollbacks, we will implement 2 web services using SOA composites and the Oracle Database adapter.

Lets assume we have a CRM System (CustomerService) and and Order Fulfillment System (OrderService). The requirement should be that an order is only stored in the Order System if the customer has been stored successfully in the CRM system. Also, if a new order is placed incorrectly, the customer should not be stored in the CRM System.

Please be aware that with a single Weblogic server domain and when not using TcpMon, you will not see any WS-AT transactions because the local optimization kicks in and SOAP will not be used….

Lets design first the CustomerService. You can find the complete source code here. (Again please click on any image to enlarge)

Create the Database User and Schema

For the 2 services we will need one schema with 2 tables:

create user wstest identified by wstest;

grant connect, resource to wstest;

connect wstest/wstest@XE

create table orders (orderid varchar2(255) primary key, quantity int, description varchar2(20));

create table customers (customerid varchar2(255) primary key, firstname varchar2(20), lastname varchar2(20))

Modify Weblogic Configuration

Create a new data source “jdbc/wstest” in the Weblogic console and enter the connection settings. Be sure to add a target Weblogic server/domain…

After that, create a new connection factory in the deployed DBAdapter and point this to the wstest data source:

Design and test of the CustomerService

First create an new SOA composite named CustomerService with one synchronous BPEL component:

Change the CustomerService.xsd to pass customerid, first name and last name as parameters:

<?xml version="1.0" encoding="UTF-8"?>
<schema attributeFormDefault="unqualified"
    elementFormDefault="qualified"
    targetNamespace="http://xmlns.oracle.com/Test_WS_AT_jws/CustomerService/CustomerService"
    xmlns="http://www.w3.org/2001/XMLSchema">
    <element name="process">
        <complexType>
            <sequence>
                <element name="customerid" type="string"/>
                <element name="firstname" type="string"/>
                <element name="lastname" type="string"/>
            </sequence>
        </complexType>
    </element>
    <element name="processResponse">
        <complexType>
            <sequence>
                <element name="result" type="string"/>
            </sequence>
        </complexType>
    </element>
</schema>

Then create a new Database Adapter on the right hand composite side:

Select the db connection and schema wstest which you have created before and select CUSTOMERS as target. Select Insert or Update as operation. After the DB Adapter wizard, create an Input Variable as inidicated below:

Your SOA Composite should now look like:

Edit the BPEL Process to insert an Invoke activity to call the newly created Partnerlink:

Use a Transform activity before the Invoke to map the input parameters to the variables of the Invoke:

You now can deploy and test the CustomerService using the SOA Console Test Page and should see a inserted row in the customers table.

Design and test of the OrderService

Repeat each step of CustomerService to create a similar OrderService. Just select in the DB Adapter Wizard the table ORDERS.

In addition to the CustomerService I have selected the genuid function as input for the orderid variable. So add an additional copy rule in an Assign activity before the Transform.

This leaves only the Order Item Description as input (for simplicity each time “one “ is assumed for the order amount).

The resulting SOA composite should look like

Deploy and the the OrderService using the SOA Console and verify that an Order has been created in the ORDERS table.

Configure TcpMon

Start TcpMon with 8081 as forwarding port and the server:port where you have deployed the CustomerService (i.e. http://localhost:7001) as target.

Testing both webservice in one atomic transaction

Create a new SOA composite CreationService using a synchronous BPEL component which calls both web services – CustomerService and OrderService – sequentially:

Insert 2 web service references in the right side of the SOA composite editor and enter the WSDL URLs of CustomerService and OrderService – but be sure to use the port where TcpMon listens (8081).

Edit the XSD of CreationService and modify it to use customerid, firstname, lastname and orderitem as input parameters.

Then edit the BPEL process, insert 2 Invoke activities, calling the Partnerlinks – first for CustomerService, then Orderservice.

Map the variables so that customerid, firstname and lastname are fed into teh input variable for CustomerService and orderitem for OrderService.

Last 2 steps are:

Insert bpel.config.transaction=RequiresNew (or Required) in the CreationService BPEL component to start the transaction when the composite is called (see Part 2).

Modify the WS-AT settings to a MANDATORY transaction flow

Now you can deploy and test the CreationService.

Try first one Order Item (a arbitrary string) with less than 20 characters. This should create a new customer and a new order.

Then try with an Order Item string longer than 20 chars: this results in an SQL exception at the OrderService call and also roll backs the CustomerService as designed:

You can verify in TcpMon that the SOAP request used WS-AT.

This concludes the hands-on parts on WS-AtomicTransaction.

Web Service Transactions Part 2: WS-AtomicTransaction with SOA Composite calling EJB-Web Service

2010-08-13T04:57:00.001-07:00

In the first part we have looked at web service transactions between JAX-WS web services without any SOA composite/component.

The next scenario where we will look at is a SOA composite calling the web service WsatBankTransferService deployed in part 1. (click on each image to display in full quality):

Enable WS-AT on the SOA domain

The installation of the 11.1.1.3 Patch Set of Oracle SOA unfortunately does not update the file policy-accessor-config.xml in the created domain or WLS instance to the required WS-AT settings.

The definition of policy interceptors is however mandatory for WS-AT to work. If you did not configure anything in policy-accessor-config.xml yourself, this means that you will need to replace your version of the file with this version. (The original file can the found here for comparison).

Rename you file to .old and copy the downloaded file to the directory. Restart the Weblogic server.

Design and Deploy the Composite

First start the Weblogic samples server and use a port different than the SOA instance. I use port 8001 for the examples WLS domain and port 7001 for the admin server where the soa domain is deployed.

Then start TcpMon – to be able to monitor the SOAP request lateron and tunnel all request from port 8081 to port 8001:

We will create a BPEL based SOA composite with the SOA designer.

Create a new SOA project and a new composite with a BPEL component. Choose a synchronous BPEL process type and name it like you want.

After that, drop a web service from the resource palette into the right part of the composite editor to create a reference. Fill in the details of the WsatBankTransferService but use the port set with TcpMon. Paste the WSDL URL of the WsatBankTransferService into the form and select MANDATORY and DEFAULT for WS-AT transaction propagation and version

Next edit the bpel process to include a Invoke activity for “CreateAccount” using the newly created Partnerlink and pass the input parameter with an Assign activity to the invoke:

The result should look like:

Now you can deploy and test the composite, right?

No, because at this point, the BPEL component would not automatically start a transaction. To achieve this, edit the composite.xml and insert the bpel.config.transaction property like:

<component name="WSATBPELClient">
<implementation.bpel src="WSATBPELClient.bpel"/>
<property name="bpel.config.transaction"
       many="false" type="xs:string">requiresNew</property>
</component>
<reference name="BankAccountService"
           ui:wsdlLocation="http://192.168.56.110:8081/WsatBankTransferService/WsatBankTransferService?WSDL">
<interface.wsdl interface="http://tempuri.org/#wsdl.interface(WsatBankTransferService)"/>
<binding.ws port="http://tempuri.org/#wsdl.endpoint(WsatBankTransferService/WSHttpBindingIService)"
              location="http://192.168.56.110:8081/WsatBankTransferService/WsatBankTransferService?WSDL"
              soapVersion="1.1">
    <property name="weblogic.wsee.wsat.transaction.flowOption"
              type="xs:string" many="false">MANDATORY</property>
    <property name="weblogic.wsee.wsat.transaction.version" type="xs:string"
              many="false">DEFAULT</property>
</binding.ws>
</reference>

At this point it does not matter if you used “RequiresNew” or “Required”, any of the two will start a new transaction because we do not pass any to the inbound “client” partnerlink.

Now you can deploy the composite to the SOA domain.

Testing and Debugging WS-Atomic Transactions

Execute the composite in the EM Test page (http://host:7001/em)

You should see a successfully completed instance and several SOAP or https calls in TcpMon. The latest should be the SOAP request to create the account – with the same WS-Coordination Context as seen in part 1:

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"
              xmlns:wsa="http://www.w3.org/2005/08/addressing">
<env:Header>
    <wsa:To>http://127.0.0.1:8081/WsatBankTransferService/WsatBankTransferService</wsa:To>
    <wsa:Action>http://tempuri.org/WsatBankTransferService/createAccountRequest</wsa:Action>
    <wsa:MessageID>urn:73432EF0A5E911DFBFAC43B9B866DC33</wsa:MessageID>
    <wsa:RelatesTo>urn:73432EF0A5E911DFBFAC43B9B866DC33</wsa:RelatesTo>
    <wsa:ReplyTo>
      <wsa:Address>http://www.w3.org/2005/08/addressing/anonymous</wsa:Address>
      <wsa:ReferenceParameters>
        <instra:tracking.ecid xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">0000Id_N1rj3n3WjLxZR8A1COuk800001X</instra:tracking.ecid>
        <instra:tracking.conversationId xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">urn:73432EF0A5E911DFBFAC43B9B866DC33</instra:tracking.conversationId>
        <instra:tracking.parentComponentInstanceId xmlns:instra="http://xmlns.oracle.com/sca/tracking/1.0">reference:50001</instra:tracking.parentComponentInstanceId>
      </wsa:ReferenceParameters>
    </wsa:ReplyTo>
    <ns3:CoordinationContext xmlns:ns3="http://schemas.xmlsoap.org/ws/2004/10/wscoor"
                             xmlns:ns4="http://schemas.xmlsoap.org/ws/2004/08/addressing"
                             env:mustUnderstand="1">
      <ns3:Identifier>urn:uuid:BEA1-02783739A26FB5C280E1</ns3:Identifier>
      <ns3:Expires>298000</ns3:Expires>
      <ns3:CoordinationType>http://schemas.xmlsoap.org/ws/2004/10/wsat</ns3:CoordinationType>
      <ns3:RegistrationService>
        <ns4:Address>http://192.168.56.110:7001/wls-wsat/RegistrationPortTypeRPC</ns4:Address>
        <ns4:ReferenceParameters>
          <wls-wsat:txId xmlns:wls-wsat="http://weblogic.wsee.wstx.wsat/ws/2008/10/wsat">BEA1-02783739A26FB5C280E1</wls-wsat:txId>
          <wls-wsat:routing xmlns:wls-wsat="http://weblogic.wsee.wstx.wsat/ws/2008/10/wsat">AdminServer</wls-wsat:routing>
        </ns4:ReferenceParameters>
      </ns3:RegistrationService>
    </ns3:CoordinationContext>
</env:Header>
<env:Body>
    <createAccount xmlns="http://tempuri.org/">wwewe</createAccount>
</env:Body>
</env:Envelope>

Debugging of all WS-AT related actions on server-side can be switched on with the java command line flag

-Dweblogic.debug.DebugWSAT=true

You will see a lot of 2PC handshake requests like for example:

<WseeWsat> <BEA-224575> <WS-AT transaction id is BEA1-45D0D4EFBBE0B5C280E1 and time to live is 299,000 for transaction Name=…

<WseeWsat> <BEA-224504> <registerOperation entered with …

<WseeWsat> <BEA-224503> <Registering Durable2PC Participant

<WseeWsat> <BEA-224538> <Durable2PC WS-AT Participant created for Address

<WseeWsat> <BEA-224539> <Prepare called for Address

<WseeWsat> <BEA-224603> <Successfully created participant port

<WseeWsat> <BEA-224602> <Durable participant port placed in cache for

<WseeWsat> <BEA-224510> <preparedOperation Xid:

<WseeWsat> <BEA-224511> <preparedOperation exited with Notification

<WseeWsat> <BEA-224546> <Commit called for Address

<WseeWsat> <BEA-224590> <About to send commit for durable participant with Xid

<WseeWsat> <BEA-224518> <committedOperation entered with Notification

<WseeWsat> <BEA-224591> <Commit sent for durable participant with Xid

<WseeWsat> <BEA-224547> <Commit call received reply COMMITTED before wait was entered for Address

<WseeWsat> <BEA-224583> <Durable participant port removed from cache

<WseeWsat> <BEA-224584> <Durable participant XAResource removed from cache

Monitoring and Management

In the SOA Control web administration frontend, you can set or change the WS-AT settings for all endpoints. Right-click on the SOA composite and select “Service/Reference Properties”:

In our case, we only see the client-side composite endpoint because we do not have a SOA on the callee side.

Under “Properties” you have access to the Transaction Flow and WS-AT Version settings.

For the JAX-WS Webservice published on the Samples Server you can see the settings in the WLS console:

Navigate to the webservicesWsatSimpleEar deployment, expand the WsatBankTransferService web service and click on it. You will see the WS-AT setting under “Configuration”, “Ports”:

Click on any operation or the whole web service to modify this:

You can also verify the WS-AT setting in the WSDL on the web service:

If you look at the WSDL then you see for every binding the attached WS-AT policy:

<operation name="createAccount">
<wsp:PolicyReference URI="#WSAT10"/>

This scenario was a single operation “CreateAccount”. In the next part we will create a sample using 2 SOA composites being called by another composite in one atomic transactions. Both callees use DBAdapter, so we can play around with rollbacks easily.

Stay tuned!

Troubleshooting

If you receive the following exception when testing the composite, then you did not copy the new policy-accessor-config.xml

Caused by: javax.xml.ws.soap.SOAPFaultException: transaction context is required to be inflowed at oracle.j2ee.ws.client.jaxws.DispatchImpl.throwJAXWSSoapFaultException(DispatchImpl.java:955) at oracle.j2ee.ws.client.jaxws.DispatchImpl.invoke(DispatchImpl.java:750) at oracle.j2ee.ws.client.jaxws.OracleDispatchImpl.synchronousInvocationWithRetry(OracleDispatchImpl.java:234) at ...

Web Service Transactions Part 1: WS-AtomicTransaction with EJB-based Webservices and WLS 11gR1

2010-08-12T08:15:00.001-07:00

To make it clear at first: I am not a fan of web service transactions. They contradict the principle of loose coupling in a SOA. But it is like with XA and 2PC – there are situations where you want to use them or where it is almost mandatory.

In SOA there have been concepts like compensation in BPEL to avoid atomic transactions spanning several SOAP calls (before they were possible and available technically) but some business requirements do not allow to commit at first and undo later on.

Also, Oracle SOA Suite optimizes local calls, so transactions were possible over multiple service invocations if they are on the same application server instance (see optSoapShortcut property in 10g)

To allow web services with SOAP using transactions like in EJB-based distributed applications the following standards have been published by OASIS (http://www.oasis-open.org/specs/):

WS-AtomicTransaction v1.2

WS-BusinessActivity v1.2

WS-Coordination v1.2

The latest version 1.2 was published in February 2009 and WS-Coordination and WS-AtomicTransaction support started in Weblogic Server 11gR1 (10.3.3) and Oracle Soa Suite 11g PS2 (11.1.1.3). WS-BusinessActivity is for long running business transactions and therefore not covered here.

With WS-AtomicTransaction (abbreviated WS-AT) it is possible to call another web service (callee) from a web service (caller) within the same transaction. The transaction context is passed to the other web service via soap in a CoordinationContext structure in the SOAP header. Some other components work together as outlined here to let the second web service participate in the transaction.

Let’s have a look at how you can setup a transaction spanning 2 or more web services.

The first scenario we will look at is when you have implemented 2 web service based on JAX-WS EJBs in Weblogic and you want to call the one from the other.

A good startup example for WS-AtomicTransaction is included in the Weblogic Server 11gR1 installation when you select custom install and include the "Server Samples” for installation. Then you can move to %WLS_HOME%\wlserver_10.3\samples\server\examples\src\examples\webservices\jaxws\wsat.

After setting the examples environment, compiling and deploying with ant you will see the Bank application:

You can enter for example local account number 100 and remote account number 100 with initial amounts of 100 USD.Then try to transfer 20 USD from local to remote account. This is done in one atomic transaction.

The application consists of one servlet which makes modifications on the local account and then calls the web service for the remote located at

http://host:port/WsatBankTransferService/WsatBankTransferService

The wsdl is available at

http://host:port/WsatBankTransferService/WsatBankTransferService?WSDL

To be able to see the SOAP request to the web service I have put TcpMon inbetween: change the following lines in WsatBankTransferServlet.java to map the TcpMon port (i.e. 8081):

public void doPost(HttpServletRequest request,HttpServletResponse response)

           throws ServletException, IOException {

    // define remote Webservice server
    String url = "http://localhost:8081";
    URL wsdlURL = new URL(url + "/WsatBankTransferService/WsatBankTransferService");

The SOAP request looks like follows – the WS-AT specific parts are contained in the CoordinationContext structure:

<?xml version='1.0' encoding='UTF-8'?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Header>
    <To xmlns="http://www.w3.org/2005/08/addressing">http://localhost:8081/WsatBankTransferService/WsatBankTransferService</To>
    <Action xmlns="http://www.w3.org/2005/08/addressing">http://tempuri.org/WsatBankTransferService/createAccountRequest</Action>
    <ReplyTo xmlns="http://www.w3.org/2005/08/addressing">
      <Address>http://www.w3.org/2005/08/addressing/anonymous</Address>
    </ReplyTo>
    <FaultTo xmlns="http://www.w3.org/2005/08/addressing">
      <Address>http://www.w3.org/2005/08/addressing/anonymous</Address>
    </FaultTo>
    <MessageID xmlns="http://www.w3.org/2005/08/addressing">uuid:a64a702b-0ce6-40fc-9b12-71a494c41520</MessageID>
    <CoordinationContext xmlns:ns3="http://schemas.xmlsoap.org/soap/envelope/"
                         xmlns:ns2="http://schemas.xmlsoap.org/ws/2004/08/addressing"
                         xmlns="http://schemas.xmlsoap.org/ws/2004/10/wscoor"
                         ns3:mustUnderstand="1">
      <Identifier>urn:uuid:BEA1-0070D4A7F05CBD76B51F</Identifier>
      <Expires>900000</Expires>
      <CoordinationType>http://schemas.xmlsoap.org/ws/2004/10/wsat</CoordinationType>
      <RegistrationService>
        <ns2:Address>http://192.168.56.110:7001/wls-wsat/RegistrationPortTypeRPC</ns2:Address>
        <ns2:ReferenceParameters>
          <wls-wsat:txId xmlns:wls-wsat="http://weblogic.wsee.wstx.wsat/ws/2008/10/wsat">BEA1-0070D4A7F05CBD76B51F</wls-wsat:txId>
          <wls-wsat:routing xmlns:wls-wsat="http://weblogic.wsee.wstx.wsat/ws/2008/10/wsat">examplesServer</wls-wsat:routing>
        </ns2:ReferenceParameters>
      </RegistrationService>
    </CoordinationContext>
</S:Header>
<S:Body>
    <ns2:createAccount xmlns:ns2="http://tempuri.org/">
      <arg0>222</arg0>
      <arg1>0</arg1>
    </ns2:createAccount>
</S:Body>
</S:Envelope>

More details regarding the implementation:

The WsatBankTransferService bean contains the following annotations to enable WS-AT:

@WebService(serviceName = "WsatBankTransferService", targetNamespace = "http://tempuri.org/", portName = "WSHttpBindingIService")
@Transactional(value=Transactional.TransactionFlowType.MANDATORY,
version=weblogic.wsee.wstx.wsat.Transactional.Version.WSAT10)

Each method of the webservice looks up the JTA user transaction (if available):

ctx = new InitialContext();
tx = (UserTransaction)ctx.lookup("javax.transaction.UserTransaction");

The servlet passes the TransactionalFeature to the web service as documented in

Oracle® Fusion Middleware Programming Advanced Features of JAX-WS Web Services for Oracle WebLogic Server – Chapter 3: Using Web Services Atomic Transactions

// Passing the TransactionalFeature to the Client
private WsatBankTransferService getWebService(URL wsdlURL) {
TransactionalFeature feature = new TransactionalFeature();
feature.setFlowType(TransactionFlowType.MANDATORY);
        feature.setVersion(Version.WSAT10);
        WsatBankTransferService_Service service =
                new WsatBankTransferService_Service(wsdlURL,
                    new QName("http://tempuri.org/", "WsatBankTransferService"));
return service.getWSHttpBindingIService(new javax.xml.ws.soap.AddressingFeature(), feature);

In the next parts we will call the WsatBankTransferService from a SOA Composite and we will span atomic transactions over multiple SOA composites.

Speeding up AIA 11g R1 Server Startup Time

2010-07-15T00:33:00.001-07:00

The Oracle Application Integration Architecture has been released recently in the updated 11g R1 Version.
The name is still coming from early times - nowadays because of its capabilities and positioning it should be named more likely Oracle Enterprise Integration Architecture....

AIA 11g R1 includes several fantastic new features as well as the upgrade to Oracle Weblogic Server 11g as a foundation.
New features include the Service Constructor and the Project Lifecycle Workbench - both increasing productivity.

If you install AIA 11g you may notice however a long startup time of the Weblogic soa server. On my VirtualBox this took almost 15 minutes. This has nothing to do with a potential overhead of AIA - it is simply a misconfiguration by the AIA installer.
Here is the cause and the solution:

Cause:
The AIA installer creates several data sources and connection pools. Open the Weblogic Console (http://localhost:7001/console):

Look into the connection pool settings of each data source and verify the value of “Login Delay” under the “Advanced” section (see bottom of following image):

The AIA installer sets 30 seconds as Login Delay for each data source. This explains the long startup time because this adds up and delays the bootstrapping.

Solution:

Change this value to 0 (zero) as a login delay is completely unnecessary with a low initial limit of very few connections. (The setting is intended to prevent a huge mass of logins to the database at the same time).

You have to restart the server. After this change the startup should be typically 4-5 times faster than before.

You may want to know how to detect the issue? First set the log level of Weblogic server to “Debug”:

Change to Servers/soa_server1/Logging/Advanced:

Set the log levels of “Log file” and “Standard Out” to “Debug” or “Info”. Restart the soa_server1.

After that you will notice the following entry in the soa_server1.out
(located in /home/oracle/Middleware/user_projects/domains/AIA_domain/servers/soa_server1/logs):

Have fun!

Oracle Application Server 10.1.3.5 Patchset Released

2009-08-19T00:05:00.001-07:00

10.1.3.5 patchset has been released for Linux x86 and Windows 32 bit (64-bit Linux coming soon).
Patch is downloadable from Metalink:
Patchset 8626084
Description PLACEHOLDER BUG TO UPLOAD AS 10.1.3.5 PATCH-SET ON ARU

One of the major new features is for example as BPEL instance migration feature, where you can “move” one running instance from one revision to another revision. More on this feature coming here soon!

Load Balancing Strategies for SOA Infrastructures

2009-08-12T12:26:00.001-07:00

When designing SOA infrastructures or Enterprise Architectures, most architects miss one component which may impact the overall system performance dramatically – the hardware load balancer.

Typically a hardware load balancer like F5 BigIP exists as a shared component for many applications and services to distribute load between multiple servers or applications and to ensure failover in case of hardware or software failure.

However, the typical setup of a load balancer is rarely suited for typical SOA services as the following example shows:

A sample real life SOA service consists of a large order system which does the backend processing of customer orders. It was implemented using Oracle SOA Suite 10g and is based mainly on BPEL processes which contain the business logic and orchestrate the backend systems like Product Catalog, Fulfillment and Billing. All the requests (orders) come from a CRM portal.

This SOA implementation is running on an 8-node Linux-based BPEL cluster running Oracle Fusion Middleware 10g. In front of these 8-nodes, a F5 BigIP is distributing the load.

Initially the load balancer setup was the following:

- static round-robin to all 8 nodes
- session affinity (persistence) for a duration of one hour

The corresponding rule for the BigIP configuration looks like

persist source_addr_1h

Let’s look at the impact of these settings:

Session Affinity (persistence) means that all requests coming from the same originating IP address will be routed by the load balancer to the same cluster node. This rule is mostly suited for typical web applications where it is desired to keep one HTTP session on the same cluster node.

However this setup has a severe negative impact for backend SOA services as we can see shortly:

The SOA order service consists of one invocation endpoint and some partnerlinks for calling the backend systems. Most of this communication is asynchronous, that means the backend systems are called via one-way requests and use callbacks to return messages back to the order system.

What now happens is that the first request to the order system comes from the CRM system, resulting in a routing to one node (say “Node A”) of the cluster. If these requests come over the load balancer, then all subsequent request coming from the CRM system will also be routed to the same cluster node, because the originating IP is equal! This results in a very poor overall load distribution. In fact only one of the 8 nodes will get most of the requests. When this node is at 100% resource utilization, the system will dramatically slow down even though the other 7 nodes are almost idle.

But it is even getting worse.

First of all, the same effect is in place for every callback from a backend system. So, if a callback (return message) is received from the Billing System, also the same node gets these answer repeatedly within one hour (though this need not be Node A, of course).

Secondly, the static round-robin algorithm does not take in effect, which state each cluster node has. So, for example if one cluster node is heavily under load, because it processes some complex orders, and this results in 100% cpu load, then the load balancer will not recognize this but route lots of other requests to this node causing overload and saturation.

In summary, a small misconfiguration of the load balancer will lead to a system which does not use the hardware of 8 nodes effectively, which will not be able to handle lots of requests and which will not scale well.

So what are the recommendations?

1. There should be no session affinity (persistence) at all for a runtime SOA system. There may be some exceptions, for example at deployment time. When deploying SOA services with multiple artifacts (for example multiple BPEL processes, WSDLs and XSDs), this should happen in general on the same cluster node first to prevent lots of replication and inconsistencies in the cluster. But this can be configured, for example by using a dedicated deployment server or setting up a virtual host as a deployment target.

2. BigIP offers more sophisticated load balancing algorithms than dumb static round-robin. For example you can use the dynamic ratio load balancing (described in Chapter "Configuring servers for SNMP Dynamic Ratio load balancing" of the BigIP Reference Guide). This algorithms uses metrics which are calculated dynamically by SNMP agents running on each node. These SNMP agents are typically included in the LINUX distribution and just need to be started on each node. The load balancer then regularly queries these agents about the values, dynamically calculates the metrics and routes the requests accordingly. This means that the distribution of requests will be proportionally to the metrics of each node for each time frame.

An example might look like

monitor type real_server {
interval 5
timeout 16
dest *.12345
method "GET"
cmd "GetServerStats"
metrics "ServerBandwidth:1.5,CPUPercentUsage, MemoryUsage,
TotalClientCount"
agent "Mozilla/4.0 (compatible: MSIE 5.0; Windows NT)
}

The configuration of the Dynamic Ratio Load Balancing needs the following settings on BigIP side (example):

Interval: 1 minute
Timeout: 10 seconds
Community Strings: "public"
SNMP Version: V2

The overall effect is that only nodes which are not under load will get requests, and that the overall distribution of load will be much more effective. All cluster nodes will be utilized and the overall system will scale well up to the limit of the all available nodes. Then, of course, when all 8 cluster nodes are at 100%, this algorithm cannot help anymore of course….

References:

- BigIP Reference Guide

- Oracle SOA Suite 11g Enterprise Deployment Guide

- Oracle SOA Suite 10g Best Practises

- Oracle Fusion Middleware 11g High Availability Guide

Enterprise Architecture Roundtable in Berlin and Frankfurt – Bookmark Your Calendars

2009-07-22T12:44:00.001-07:00

On September 2nd, 2009 in Frankfurt and September 3rd, 2009 in Berlin, Oracle will organize an Enterprise Architecture Roundtable to discuss trends in infrastructures and methods for modern IT platforms. Main focus is to facilitate the exchange of Enterprise Architecture knowledge and experience between lead architects of our main customers. You are welcome to join. Register directly here:

2. September 2009 – Frankfurt

3. September 2009 - Berlin

Running Oracle Soa Suite 11g on OpenSuse 11.1 – Part 2

2009-07-17T03:19:00.001-07:00

PART 2 – Starting SOA Suite and Creating a First SOA Composite in JDeveloper 11g Version 11.1

This continues Part 1 which can be found here.

After installation we need to take one steps before starting the servers:

Configure a boot.properties file in the soa_domain root directory:
This is simply a text file with 2 lines:
username=weblogic
password=welcome1

After first system start the password entry will be encrypted.

Starting SOA Suite contains of 2 servers:

an AdminServer (running WebLogic Console and Grid Control Console)
a ManagedServer soa_server1

At runtime, only the ManagedServer is required.

Now we can start the servers with

<wl_home>/user_projects/domains/soa_domain/startWebLogic.sh

Test if you can access the WebLogic console at http://<host>:7001/console

After completion, start the Managed Server:

<wl_home>/user_projects/domains/soa_domain/startManagedWebLogic.cmd soa_server1

Check if you can access the soa infra test page at :8001/soa-infra">http://<host>:8001/soa-infra

Now you can also login to Grid Control at :7001/em">http://<host>:7001/em

Notice the Linux System Resources: Altogether we use only 2,5 GB of RAM (including Linux…!)

Next install and run JDeveloper 11.1 (I used the generic JAR installer) and create an Application Server Connection:

From the About Menu, Check for Updates and install the SOA Composite Editor (this is NOT included in the default installation!):

Now you are ready to create your first application:

New Application
New Project, for example a SOA Composite with a BPEL process:

With Project –> right click you can Deploy the composite to the SOA server:

Running Oracle Soa Suite 11g on OpenSuse 11.1 – Part 1

2009-07-15T14:50:00.001-07:00

PART 1 – Installation of Oracle Database, Weblogic Application Server and SOA Suite 11gR1.

Not only the version numbers match ;-) Also the products – OpenSuse 11.1 and Oracle Fusion Middleware 11gR1 form an excellent couple even though not officially certified. While Oracle Enterprise Linux is the officially certified version, it lacks more recent kernel support which means also that WLAN drivers might not work for example. So if you are looking for a lightweight Open Source Linux based installation (as development environment) of SOA Suite 11gR1 here is the way how I did it (Installation on Toshiba Tecra M9 Notebook with 4 GB RAM with Gnome desktop).

1. Be sure to install OpenSuse 11.1 with the development tools and libraries and with 32-bit support for gcc, glibc and glibc-devel.

2. Create user oracle and group oracle. Login as user oracle.

3. Install Oracle Database 11.1.0.6 Standard Edition and create a default OLTP database with the creation assistant. Be sure to use character set AL32UTF8. I used an SGA size of 512 MB with autoextend=on.

4. Install Weblogic Server 11g (10.3.1) – I used the generic jar installer.

5. Install the 11gR1 Database Repository Creation Utility and start it from <install>/bin/rcu

To save space, I created single tablespaces “DEV_SOA” and “TEMP” for all repository schemas:

6. Install the Oracle SOA Suite 11gR1 into the recently created WebLogic (BEA) Home.

7. Next, execute <WLS_HOME>/common/config.sh to create a new WLS domain for SOA.

First check if the Sun JDK has been set in commEnv.sh as JAVA_HOME – otherwise set the <WLS_HOME>/jdk160_11 as JAVA_HOME.

Set the database connections to the ones created with RCU:

Finally you are done!

Troubleshooting:

When you receive an error at Database linking in ins_client.mk at database installation time: Skip this step, complete the DB installation and execute <ORACLE_HOME>/bin/relink client after installation.
If executing runInstaller fails, then include the option –jreLoc <path to JDK160_11> on the command line

JRockit JVM now certified for SOA Suite 10.1.3.4

2009-05-07T00:54:00.001-07:00

Oracle JRockit (formerly BEA JRockit) is a well recognized JVM which has many advantages over Sun JVM. For example it overcomes the usual difficulties with PermGen space limitations of the Sun JVM, just to name one. In benchmarks in a real complex SOA project we have experienced performance advantages of JRockit over Sun on Linux of more than 50%!

Since today, JRockit 1.6 (R27.6.3) is now certified for SOA Suite 10.1.3.4 on Linux and Windows!

Please see the updated Oracle Fusion Middleware 10gR3 Certification Matrix.

More on monitoring and performance tuning with JRockit and its superior Mission Control tool soon…

Using Oracle JMS Adapter – Part 1: Publish from BPEL to AQ via Partnerlink Invoke

2009-04-30T02:28:00.001-07:00

Using the Oracle JMS Adapter is the standard way to inititate asynchronous communication to partner systems, for example JMS-based services, to databases using Oracle AQ or to IBM Websphere MQ.

A complete sample (for SOA Suite 10.1.3.4) demonstrates the steps required for implementing a fire-and-forget (one-way) invoke to an AQ queue on an Oracle database:

1. Create the following datasource and connection pool (in oc4j_soa/config/data-sources.xml)

<managed-data-source name="SensorTestDS" connection-pool-name="SENSOR_CONNECTION_POOL" jndi-name="jdbc/SensorTestDS"/>

<connection-pool name="SENSOR_CONNECTION_POOL">
<connection-factory factory-class="oracle.jdbc.pool.OracleDataSource" user="jmsuser" password="jmsuser" url="jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=tcp)(HOST=skoser-de.de.oracle.com)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=DB10G.SKOSER)))"/>
</connection-pool>

(Note: The data source is named “Sensor..” because part 2 shows how to use this from a bpel activity sensor)

2. Enter resource provider in oc4j_soa/config/application.xml:

<resource-provider class="oracle.jms.OjmsContext" name="SendorTestRP">
<description>Sensor JMS/AQ</description>
<property name="datasource" value="jdbc/SensorTestDS"/>
</resource-provider>

The value of the datasource property must match the JNDI lookup (“jndi-name”) of the datasource in data-sources.xml.

3. Create database objects: create a queue named “JMS_OUT” and a queue table “JMS_OUT_TABLE”.

/* Execute as SYS */
CONNECT SYS AS SYSDBA
DROP USER jmsuser CASCADE ;

GRANT connect,resource,AQ_ADMINISTRATOR_ROLE TO jmsuser IDENTIFIED BY jmsuser ;
GRANT execute ON sys.dbms_aqadm TO jmsuser;
GRANT execute ON sys.dbms_aq TO jmsuser;
GRANT execute ON sys.dbms_aqin TO jmsuser;
GRANT execute ON sys.dbms_aqjms TO jmsuser;

connect jmsuser/jmsuser;

begin
sys.dbms_aqadm.create_queue_table(
queue_table => 'JMS_OUT_TABLE',
queue_payload_type => 'SYS.AQ$_JMS_MESSAGE',
sort_list => 'PRIORITY, ENQ_TIME',
compatible => '10.0.0',
primary_instance => 0,
secondary_instance => 0,
storage_clause => 'tablespace users pctfree 10 initrans 1 maxtrans 255 storage ( initial 64K minextents 1 maxextents unlimited )');
end;
/

begin
sys.dbms_aqadm.create_queue(
queue_name => 'JMS_OUT',
queue_table => 'JMS_OUT_TABLE',
queue_type => sys.dbms_aqadm.normal_queue);
end;
/

begin
dbms_aqadm.start_queue( queue_name =>'JMS_OUT' ,enqueue => true ,dequeue => true );
end;

5. Create a new connection factory entry in JMSAdapter oc4j-ra.xml
(oc4j-soa/application-deployments/default/JMSAdapter/oc4j-ra.xml)

<connector-factory location="eis/Jms/11gAQ" connector-name="Jms Adapter">
<config-property name="connectionFactoryLocation" value="java:comp/resource/SensorTestRP/QueueConnectionFactories/QCF"/>
<config-property name="factoryProperties" value=""/>
<config-property name="acknowledgeMode" value="AUTO_ACKNOWLEDGE"/>
<config-property name="isTopic" value="false"/>
<config-property name="isTransacted" value="false"/>
<config-property name="username" value="jmsuser"/>
<config-property name="password" value="jmsuser"/>
<connection-pooling use="private">
<property name="waitTimeout" value="180" />
<property name="scheme" value="fixed_wait" />
<property name="property-check-interval" value="5" />
<property name="inactivity-timeout-check" value="all" />
<property name="abandoned-connection-timeout" value="20" />
<property name="autoCloseSession" value="" />
<property name="inactivity-timeout" value="120" />
</connection-pooling>
<security-config use="none">
</security-config>
</connector-factory>

The string after “java:comp/resource/” must match the resource provider name in application.xml.

IsTransacted must be set to false for outbound JMSAdapter (see Metalink Note 418884.1 ).

6. Restart oc4j

7. Design a bpel process using the JMS Adapter:

Create a default async bpel process and insert one assign and one invoke activity.

Create a new partnerlink using the “Define Adapter Service” icon:

Define a db connection to the jmsuser schema:

Continue with the Adpater service creation

The Destination Name above must contain

java:comp/resource/SensorTestRP/Queues/JMSUSER.JMS_OUT

We want to publish employees to the queue so copy the addemployees.xsd form here to the BPEL project (in directory bpel) and reference it:

The resulting bpel process should look like this:

You can download the BPEL Process project for JDeveloper 10.1.3.4 here.

8. Deploy the BPEL process to the default domain.

9. Test the process from the BPEL Console:

Verify in the database that the message has been written to the JMS_OUT_TABLE:

SQL> select count(*) from JMS_OUT_TABLE;

COUNT(*)
----------
1

Next time, we will cover using sensors with JMS Adapter and then Native MQ Adapter…..

Troubleshooting:

If you receive an Orabpel-12101/ORABPEL-12102 ERRJMS_TRX_COMMIT/ERRJMS_TRX_ROLLBACK then you have not set isTransacted to false for outbound JMS Adapter invocation….

Governance: Installation Tip for Oracle Service Registry on Weblogic 10.3

2009-03-19T05:57:00.001-07:00

If you follow the Installation Guide for installation of Oracle Service Registry on Weblogic Server 10.3. you might run into a problem, that you cannot login into the Registry Console.

You might get an error like

ERROR: uddi_security_v3.com.systinet.uddi.v3.InterceptorProxy - No Configuration was registered that can handle the configuration named NamePasswordAN

EXCEPTION: No Configuration was registered that can handle the configuration named NamePasswordAN

The solution is described in Metalink Note 785097.1: Installing Oracle Service Registry 10.3 on Oracle Weblogic Server 10.3

You have to include the login module for weblogic in the JAVA_OPTIONS:

set JAVA_OPTIONS=%SAVE_JAVA_OPTIONS% -Djava.security.auth.login.config=%OSR_HOME%\conf\jaas.config

Be sure to replace %OSR_HOME% with your OSR installation directory…..