SOA, EAI, BPM, Oracle, Cordys

Struggling with File Connector

2011-09-28T20:46:00.000+02:00

Introduction
Today I was struggling with the File Connector in combination with CARS authentication and the multi-tenancy within Cordys. I really like the multi-tenancy concept, but some features in Cordys are not mature yet.

Context
We have a project with a lot of Organizations using the same BPM processes. These processes are deployed within the System Organization. The Organizations (the number can be up to 2000 !) are provisioned using Cordys Cloud Provisioning (CCP). The (Cordys) Users are authenticated through WS-Security within the soap headers.
In one of the processes we want to do a call within the context of another Organization.

Options
When I read the community blogs and asked some people they came up with

1 - UDDI Interceptor
In this case you call a webservice of Cordys through a UDDI connector and use an interceptor to add the WS-Security header. The user authenticated has been put within a default Organization, so this way the call is done within the context of that users Organization.
Note: There is a lot of information within the community to write such an interceptor.
Note2: It is not a very elegant solution, because you would expect this to be configurable within the UDDI connector itself (just like any other EAI/BPM tool)

However there was a catch, so that this option did not work. All processes are deployed within System and the BPM process is deployed within a BPM Service Container with namespace N and operation OP. When this method is to be called from within a UDDI connector (and this UDDI connector is also in System Organization), this is done with the same namespace N and operation OP.
So Cordys has actually two Service Containers where to send the message to

2 - File Poller
Because the call has to be done after a file has been placed on some directory, the file poller would be another option. In this case the trigger of the file poller would be a call to the BPM webservice with another Organization as context.

So the XML Poller configuration looks something like:

However this time you ran into the problem that the trigger does not insert a WS-Security header.
(see also http://code.google.com/p/cordysfilecon/issues/detail?id=33)

The way this could be solved was by using another soap request:

Thanks, Roger van de Kimmenade

Provisioning: Defining an Application

2011-09-12T20:53:00.000+02:00

Introduction

This blog item shows an example on how to define an application that can be used by other Organisations. In this example we have a simple Cordys BOP4 setup. We have only the Default Cluster. In this example the ISV packages are made and it will show how to define the application within CCP. It will show how easy this is within CCP. After this you are able to Provision organisations to use applications within the Cloud.
An introduction to CPP was already given here.

Define the Business Partner
First you will have to define a business partner that developed the ISV. This business partner is needed when defining the Application.

Goto System Organization
Open Global Business Partners

Select +-sign
Fill in the details

Click Save

Define the Application

Note: A CCP application is NOT a synonym for a ISV package. An application can encompass several packages and a package can be used for the definition of several applications.

Goto System Organization
Open Global ApplicationsClick +-sign
Fill in the Application details:
Application id: com.ciber.utilityservicesApplication name: CIBER Utility ServicesNote: Within ISV the CIBER Netherlands can be picked

Click Save
Now the following Tabs are shown at the bottom

The Roles are used to define all the roles that are available for the application.

The X-forms are used for GUIs that are used during provisioning. Here you can add for example an input GUI each time a new Organisation is provisioned. For example to ask for extra attributes.

The BPMs are used during provisioning.

Select Roles tab
Select the first Role as Role Id

Tip: It is advised to put functional roles in a separate ISV package, maybe in a later blog item i will come back to this.

Fill in further details for the Role
Click Save
Do the same for other Roles

Note: When an application is assigned to an Organisation each Role can be used within that Organisation. You have to be aware of this when defining the application. Sometimes you do not want some Organisations to have a certain Role. In that case it is best to define a separate Application.

After that you have to define on which cluster the application runs.

Select Installed on clusters tab
Click +-sign
Select the Default Cluster
Click Save
Click Validate (This is important, because then the state of the application becomes In use)
Click Back
Click Save

Before the application can be used it must be added to a Channel. A Channel in this case is the Global Channel that is defined when installing Provisioning.

Add Application to Default Channel

Goto System Organization
Open Global Channels

Select Collection tab
Open Collection

Click on +-sign to add an application to this collection
Within Application id select the com.ciber.utilityservices application
Click Save

Now you can assign Applications to Organisations.

Cordys - Designing short lived processes

2011-07-08T14:20:00.004+02:00

When designing short lived processes you have to watch out.An example short lived process with several different types of activities.

Within a process you can use several Activities:

Calling a subprocess within the same package
Calling a subprocess within another package
A webservice call:
- A runtime reference webservice call
- A UDDI webservice call
- A webservice call (drag-and-dropped from another package)

Execution
The short lived process is executed within a BPM service container and uses a thread for it. It holds the http connection for this call.

However be aware that the runtime behavior of the 3 activities is different.

Another BPM process can be executed within the same service container and on the same thread
Same as 1)
A webservice call is done through the Bus again and Cordys does not know where the call will be executed eventually. Note that this can be the SAME service container as the one, where the current short lived process is executed. Another thread is needed and eventually (when doing more such webservices calls) the service container may use all threads. This can cause that the client gets a timeout and the next calls cannot be executed.

A way to avoid case 3) is that extra service containers are defined to handle subsequent calls.

For example:
When you have 2 ISV packages that each contain different functionality, you can decide to have separate service containers at runtime. Another advantage is that you can stop the service container of one component without stopping the other service container, so that components can be stopped.

Conclusion
Be aware of the runtime consequences when designing short-lived processes.

Cordys: The value is not set for the parameter number

2011-06-21T20:16:00.001+02:00

Today I ran into the following problem:

<log4j:event logger="com.cordys.xds.internal.transaction.query.XQYQuery" timestamp="1308643015133" level="ERROR" thread="Monitor Stream Reader Thread">

<log4j:message><![CDATA[Failed to execute query. Error: + '

<error

TYPE="Enumeration">

<elem>Error :com.microsoft.sqlserver.jdbc.SQLServerException: The value is not set for the parameter number 1.</elem>

</error>

']]></log4j:message>

<log4j:MDC><![CDATA[host=cvzlaca004 processid=1314]]></log4j:MDC>

<log4j:throwable><![CDATA[com.cordys.xds.transaction.exception.DatabaseQueryException: Failed to execute query. Error: + '

<error

TYPE="Enumeration">

<elem>Error :com.microsoft.sqlserver.jdbc.SQLServerException: The value is not set for the parameter number 1.</elem>

</error>

Trace:

at com.cordys.xds.internal.transaction.query.XQYQuery.executeQuery(XQYQuery.java:221)

at com.cordys.xds.internal.branch.BranchDataCache.prepareParentBranchHierarchy(BranchDataCache.java:325)

at com.cordys.xds.internal.branch.BranchDataCache.loadCacheValueInternal(BranchDataCache.java:134)

at com.eibus.util.cache.Cache.loadCacheValue(Cache.java:375)

at com.eibus.util.cache.Cache.get(Cache.java:264)

at com.cordys.xds.internal.branch.BranchDataCache.get(BranchDataCache.java:217)

at com.cordys.xds.internal.branch.BranchManagerImpl.getBranchDataValue(BranchManagerImpl.java:443)

at com.cordys.xds.internal.branch.BranchManagerImpl.getBranch(BranchManagerImpl.java:421)

at com.cordys.xds.internal.branch.BranchManagerImpl.getBranch(BranchManagerImpl.java:411)

at com.cordys.xds.internal.service.XDSServiceImpl.isValidBranch(XDSServiceImpl.java:353)

at com.cordys.xds.internal.service.XDSServiceImpl.getServer(XDSServiceImpl.java:329)

at com.cordys.task.xds.TaskUtil.createTaskBranch(TaskUtil.java:168)

at com.cordys.applicationconnector.task.TaskConnector.open(TaskConnector.java:56)

at com.eibus.soap.Processor.open(Processor.java:752)

at com.eibus.soap.Processor.startProcessor(Processor.java:1036)

at com.eibus.soap.ProcessStreamsHandler.startProcessor(ProcessStreamsHandler.java:681)

at com.eibus.soap.ProcessStreamsHandler$StreamReader.run(ProcessStreamsHandler.java:353)

]]></log4j:throwable>

<log4j:locationInfo class="com.cordys.xds.util.XDSLogger" method="exception" file="XDSLogger.java" line="242"/>

</log4j:event>

The reason for this problem is that the user did not have enough privilage on the database to execute this query. So Cordys needs to have db_writer, db_reader AND db_owner rights on the database!

It took me half a day to figure this out, so hopefully this helps other Cordys developers !

Introduction to Cordys Cloud Provisioning

2011-06-19T20:02:00.002+02:00

Introduction

In my current Cordys project I am using the Cordys Cloud Provisioning (CCP) functionality of Cordys. I am very excited about this produkt and this blog item I will share my knowledge about it.

First I will explain something about provisioning.

What is Provisioning?

Provisioning is the process of managing Organizations (also called Tenants) and Users to let them use Applications. It manages the usage of applications that can be used as SaaS applications by tenants. It can be used to create new business models. Cordys has made a separate package for this, on top of BOP4 called CCP.

With CCP it is possible (among others):

Managing Cordys-Applications and non-Cordys applications
Managing of Tenants
Managing of Users and Roles
Keeping track of the usage of the Applications (Metering)

Some Terminology

The next picture shows the Cordys-ecosystem.

A Tenant makes use of SaaS applications that are delivered by a PaaS. These applications are developed by Apps Provider (ISV).

A Tenant Admin manages Users and Applications for a tenant. The End User is a user that uses the applications. A Sales Channel is a channel that sells applications for ISVs to other tenants.

Cordys is multi-tenant in the core

The nice part of Cordys is that the Cordys core is Tenant aware. This means that all usage of applications is done in the context of a tenant (in Cordys this is an Organization). Applications developed with Cordys must be made tenant aware. The result is that application data will have tenant info associated with it. In fact an Organization can be seen as a "container" with its own users, applications and security context. This way tenants can not use data of other tenants. Users can only access an organization if it has a Role in that Organization.

CCP Applications

Applications can be used by tenants and are defined in Cordys with CCP. Applications are defined with the use of ISV packages. For each Application the available Roles can be defined. You can define provisioning processes, properties and custom UIs that are being executed in case an organizations and/or users take a subscription.

Some example screens when defining a new Application through CCP.

Defining Roles.

Conclusion

Cordys is the only BPM produkt I know of, that has SaaS features within the core of the produkt. CCP is a fairly new addon to the Cordys BOP4 platform, but I expect a lot from it.

Address already in use error

2011-06-17T16:25:00.002+02:00

During installation of Cordys BOP4 (CU17) we ran into the following problem.

I tried to change the client connection point address of the WebGateway, but still no success.

Eventually it appeared to be an Apache Config problem.

In <Apache_Install_Dir>/conf/extra/httpd-mpm.conf there is a setting.

# worker MPM

# StartServers: initial number of server processes to start

# MaxClients: maximum number of simultaneous client connections

# MinSpareThreads: minimum number of worker threads which are kept spare

# MaxSpareThreads: maximum number of worker threads which are kept spare

# ThreadsPerChild: constant number of worker threads in each server process

# MaxRequestsPerChild: maximum number of requests a server process serves

ServerLimit 1

ThreadLimit 200

StartServers 1

MaxClients 200

MinSpareThreads 25

MaxSpareThreads 200

ThreadsPerChild 200

MaxRequestsPerChild 0

</IfModule>

The important setting here is StartServers. This value MUST be "1".

ORA - Oracle SOA Foundation

2011-02-22T11:32:00.002+01:00

Service Oriented Architecture

Issues todays IT:

Heterogeneity: no interoperable interfaces, now we have web services
Business Focus
Agility, reuse and cost reduction

Benefits of SOA

Reuse of Services (at the business unit level)
Agility: respond quickly to changing business requirements
Simpler integration (by using standards)

Architectural Principles

Formal contracts (Thomas: Contract Standardization)
Loose Coupling
Abstraction
Reusability
Autonomy
Statelessness
Discoverability
Composability
(Durability)
(Interoperability)
(Standards compliance)

(First all from Thomas Erl's books, last three added by Oracle)

Service Definition principles

Service encapsulation - A Service must perform a complete unit of work
Service loose coupling - Services maintain a relationship that minimizes dependencies between them
Service contract - Services adhere to a contractual agreement, as defined collectively by one or more Service description documents
Service abstraction - Beyond what is described in the Service contract, Services hide logic from the outside world
Service reusability - Logic is divided into Services with the intention of promoting reuse
Service composability - Collections of Services can be coordinated and assembled to form composite Services
Service autonomy - Services have control over the logic they encapsulate
Service optimization - All else equal, high-quality Services are generally considered preferable to low-quality ones (“quality” in this context is referring to business value, reuse, and other chosen measures of value)
Service discoverability - Services are designed to be outwardly descriptive so that they can be found and assessed via available discovery mechanisms

Oracle's definition of a Service

"A Service is a means of packaging reusable software building blocks to provide

functionality to users, applications, or other Services; it is an independent,

self-sufficient, functional unit of work that is discoverable, manageable, measurable,

has the ability to be versioned, and offers functionality that is required by a set of

consumers. A Service may be shared, which means that the function offered by the

Service is intended for multiple consumers, some known, and others that have not yet

been identified."

A Service is comprised of three parts: the implementation (deployed code and

configuration of infrastructure), the interface (means by which the Service is invoked),

and the contract (a description of what the Service provides and its constraints)

Architectural Models

SOA Logical Architecture

Layering to achieve separations-of-concerns.

Service Layers

Presentation
Business Process and Business Activity

While Business Activity Services focus on static business functions, Business Process Services provide dynamic workflow orchestations of business functions. Business Services represent operations that either have a high likelihood of reuse, or a significant value to the organization.
Data (CDM)

Data Services are used to access data from various sources using many different technologies, and present data in a business-friendly form. Data Services offer a way to aggregate, transform, and synchronize data from multiple sources. They expose data in a format that best supports business Services and composite business applications.
Connectivity

They establish connectivity with legacy applications that do not inherently provide Service oriented interfaces. Services exposed at this layer are not intended to reflect business context in any way. Being context-neutral, they can be used in many different business scenarios.
Utility (Not containing business logic)

Services generally perform infrastructure-related functions, such as security (credential mapping, access control, auditing), logging, notification, policy lookup, transaction watermarking, etc. This classification of IT functions is often referred to as "cross-cutting" and are, therefore drawn perpendicular to the other services in thisdiagram;

Service Providers and Service Consumers

Service Provider

It is important to note that providing a Service is more than simply providing an interface to invoke a business function. The Service provider must ensure that qualities of Service are stated, offered, and adhered to. This is especially important for Services of an enterprise-class nature, i.e., Services advertised for use across departmental boundaries and/or Services that are leveraged by mission-critical solutions. Therefore the principles of Service enablement must be followed.

Architectural Principles

1. An interface must be provided that conforms to the reference architecture

2. A contract must be provided as specified in the reference architecture

3. Security policy must be defined and enforced

4. Versioning strategy must be adhered to

5. The Service must conform to infrastructure rules for discovery, management, monitoring, and governance

6. The Service must be classified according to layers and definitions of the reference architecture

7. The Service provider must ensure that the Service will satisfy the aggregate specifications of all related usage agreements

Service Consumer

Service consumers are, of course, consumers of Services, but more importantly they do not offer an SOA capabilities nor are they required to conform to SOA principles (assuming they are not also Service providers). Service consumers may be end-users, composite applications, or other systems in purely system-to-system interactions.

Composite Applications

Composite Applications is a term used to define applications built primarily of Services. These Services may belong to any of the Service layers previously described. Common examples of Composite Applications include portals and BPM processes.

Unlike Composite Applications, Composite Services do not provide complete, self-contained functionality to an end-user, but instead provide Services to other Services, systems, or applications. Composite Services are comprised of other Services and are hence, Service Consumers; since they provide Services they are also Service Providers.

Service Scope

Multi enterprise (Exposed Services)
Enterprise Wide (only used within the Enterprise)
Intra-line of Business (LOB) (Only used within a LOB)
Intra-Application (Only used within an Application)

Service Meta Model

Contract

A contract describes the Service in human-readable terms, enabling a solution designer to determine its capabilities and characteristics. It includes both functional and non-functional terms (such as invocation protocols, security requirements, semantics, transaction requirements, invocation style, quality of Service).

Service policies are reusable, composeable descriptions of behavior used to specify Service contracts.

There are various policy types:

Compliance

Compliance policies can take the form of industry or enterprise standards, or regulatory and other legal mandates (example WS-I).

Quality-of-Service

Mostly non-functional like availability, performance, transaction integrity. WS-ReliableMessaging, WS-Transactions, WS-Coordination, WS-AtomicTransaction and WS Business Activity Framework, WS-Security

Message Exchange Pattern (MEP)

Examples are request-response, one-way, asychronous (WS-Addressing), pub-sub, broadcast. Other considerations: transactional requirements, only once or idempotent.
Security

Following aspects play a role:

message level security (WS-SecurityPolicy)
transport level security
authentication
authorization.

Other security standards specified by OASIS:

■ WS-Security Core Specification;

■ Username Token Profile;

■ X.509 Token Profile;

■ SAML Token profile;

■ Kerberos Token Profile;

■ Rights Expression Language (REL) Token Profile;

■ SOAP with Attachments (SWA) Profile.

Interface and Messaging
Interface often soap over HTTP, because:

Platform independant, vendor support, standards support, firewall penetration.

Disadvantages can be: verbose, reliability and security due to lack of security standards.

Implementation

An implementation will often consist of deployable code as well as infrastructure configurations, security policies, management agents, etc. - all parts working together to fulfill the contract. The Service Infrastructure side typically provides the Service enablement capabilities for the implementation. These capabilities may include, exposing the interface as a Web Service, handling SLA enforcement, security, data formatting, and others. Service infrastructure should be utilized when possible, as it reduces the burden on Service providers, from an implementation standpoint.

Separation of Concerns

There must be a clear separation of concerns in terms of what they do from how they are used. Services should be written to accomplish their function regardless of what protocol is used to invoke them, where they physically exist, or on what type of hardware or operating system they run on. This provides for maximum reuse by allowing access through multiple types of interfaces. It also provides greater versatility in how they are deployed and what underlying technologies are used.

Principle: Services must not be tied to any particular underlying technology, delivery channel, or physical location.

Service Versioning

Following the loose coupling principles of SOA, the impact to consumers should be minimized. For this reason changes should be handled as versions, ideally supporting the ability to have multiple versions running concurrently.

Principles

1. Services must support multiple concurrent versions.

2. Service providers must be able to release new versions into production without waiting for consumers to certify on them.

3. Consumers must be able to test and certify on new Service versions before switching to the new version in production.

Technologies and Standards

SOA must be standards-based in order to achieve interoperability. Also for given Enterprise, technologies (in the form of platforms), tools, and engineering practices must adhere to selected standards.

Standards may originate from various sources:

■ IT industry standards (e.g. W3C XML, OASIS WS-*);

■ Business industry standards (e.g. HL7, …);

■ Enterprise standards (engineering standards).

The primary benefits of adopting standards are interoperability, technology, independence, and rapid integration.

Core SOA related standards recommended are:

■ Java API for XML Web Services (JAX-WS)

■ Business Process Execution Language (BPEL)

■ A collection of Web Services standards (WS-*)

■ Simple Object Access Protocol (SOAP) with attachments and Message Transmission Optimization Mechanism (MTOM) (implemented as SOAP with Attachments API for Java (SAAJ)

■ HyperText Transfer Protocol (HTTP)

■ Universal Description, Discovery and Integration (UDDI)

■ Service Component Architecture (SCA)

■ Apache's Web Service Invocation Framework (WSIF)

■ Web Service Interoperability (WS-I)

Industry specific standards are:

■ HL7 (XML messaging standard for Healthcare)

■ HRXML (for support a variety of business processes related to human resource management)

■ Telecoms IMS (IP Multimedia Subsystem, 3GPP Technical Specification TS 22.228)

architectural strategy and standards for decoupling of the technical and organizational constraints of former telecommunications strategies)

■ OASIS ebXML work, such as, CPPA (Collaboration Protocol Profile and Agreement) which describes how trading partners engage in electronic business collaborations through the exchange of electronic messages

Standards for SOA can be categorized into five areas of concern:

1. Orchestration and composition;

2. Description and discovery;

3. Messaging;

4. Implementation.

5. Management

SOA Implementation Architectural Styles

REST

HTTP provides a stateless, platform-independent protocol for the exchange of

information over a network; it is a standard for a request/response dialogue between

clients and servers.

From the definition of HTTP (or more accurately its principle author, Roy Fielding) an architectural style emerged called Representational State Transfer (REST). Core to the concept of REST is a resource which it defines as any item of information or capability addressable on a network by a global identifier (its URI in HTTP terms). The REST architectural style describes any simple interface which transmits domain-specific data over HTTP without involving higher-level messaging, such as, SOAP.

Support for REST is also starting to appear in Web Services standards: WSDL 2.0

RPC

RPC is a means of calling a method or function on another node in a distributed computing environment.

Early examples of standardized RPC's included Sun's RPC, Microsoft's DCOM, Java's RMI, OMG's SOA, CORBA, ODBC/JDBC, and SQL*Net. More recent RPC's include XML-RPC, and HTTP.

Web Services Architecture

The World Wide Web Consortium's (W3C) Web Service Architecture (WSA) provides a definition of Web Services:

"A Web Service is a software system identified by a URI [RFC 2396], whose public interfaces and bindings are defined and described using XML. Its definition can be discovered by other software systems. These systems may then interact with the Web Service in a manner prescribed by its definition, using XML based messages conveyed by Internet protocols."

Web Services Standards

The second generation of Web Services specifications (commonly referred to as WS-*) has a large focus on enabling QoS within an SOA environment, which is one of the reasons Web Services has become the primary way to expose Services.

Infrastructure

Service Enablement

Service Enablement is a key differentiator of Service implementation (as opposed to application implementation).

Service enablement is

responsible for the following aspects of a Service:

■ Configuration based logic

– Routing

– Transformations

– Versioning Support

■ Composition

– The ability to compose a new Service through the composition of two or more

Services.

■ Change Control

– Audit support

– Configuration aggregation

– Configuration Rollback support

– Real-change application. Changes are applied without requiring a

redeployment or server restart.

■ Throttling

– Thread management

– Priority based resource management

– Consumption monitoring

■ Security

– Centralized Policy Management

– Distributed Enforcement

– Security Standards (WS-Security, WS-Policy, SAML, SSL, etc).

■ Interface exposure

– Using accepted interface protocols, transports, message types, and standards

– Service Discovery (published to a registry and/or repository)

– Service Definition (creation of a contract)

Service Discovery

There are two sides to consider regarding Service discovery: design time discovery and run time discovery.

Service Mediation

Among their many capabilities are:

■ Translate, or transform request and response messages

■ Accept requests via one transport or protocol and forward them on using a different transport or protocol

■ Route messages based on content within the request message (Content-based routing)

■ Route messages based on security policies

■ Translate (map) security credentials between different users/groups/roles or between different credential types

■ Add or remove security measures such as encryption and certificates

■ Invoke multiple Service providers as part of a single Service request

■ Audit and/or log requests

■ Deny requests based on access policies (SLAs, Usage Agreements)

■ Capture response time metrics and usage metrics

■ Monitor and report on error conditions

Architectural principle:

Services must be loosely coupled, e.g. they must always be accessed via an intermediary. Consumers must not directly access Service provider endpoints.

Packaging and Deployment

Service infrastructure brings its own complexities to the construction of deployable packages of Service capabilities.

Common types of deployable artifacts in a SOA environment generally include the following:

■ WSDL, XML Schema, XSLT, Xpath, various rules definitions, WS-* docs

■ Code (in the form of JAR, WAR, EAR)

■ Directory definitions

■ Access controls

Services deployment

The approach, recommended by Oracle, to Services deployment is the separation of Services from applications and the grouping of Services with similar availability needs. To accomplish this, projects must be logically divided into shared and non-shared parts. The shared parts become Services, while the non-shared parts become ordinary applications.

The illustration shows an application cluster used to host ordinary applications, and a Services cluster hosting Services. The clusters connect through the Services bus, allowing access to Services by the applications. Redundant hardware is used in both clusters to provide high availability.

Important considerations for grouping Services:

■ Ownership of resources. This includes hardware and software ownership, and responsibility for management.

■ Technology. Different technologies used for Service implementation will require different clustering strategies and may require different hardware platforms.

■ Service Level Agreements. Services that demand high availability and low downtime or maintenance windows should be separated from Services that do not share these requirements.

Federated

Federation also applies to the level of autonomy each domain has with respect to processes and Services. Processes in one domain may leverage processes that are offered as shared Business Process Services in another domain, as illustrated above. Services that represent automated business activities (tasks) within a process may be deployed in the local domains. In this scenario each domain retains autonomy in terms

of how its processes are written, which tasks are fulfilled by Services, and how the Services are offered (interface, security, implementation specifications). It is most useful when a "black box" approach is desired, i.e., details of sub-process flows including current status, execution logic, etc., do not need to be known or managed at the enterprise level.

Another option for process federation is shown below. This method involves defining processes and sub-processes at the enterprise level. The Services that fulfill automated tasks may be offered in different locations, and accessed in different ways. This method is preferred when entire end-to-end process management and

monitoring is needed. Each process and sub-process can be controlled at the enterprise level, while Services are managed at the domain level.

Social Media a necessity for Consultants ?

2011-02-08T20:57:00.001+01:00

Introduction

I work as a Enterprise Application Integration / Service Oriented / Business Process Management consultant at a company. I started using Blogger, LinkedIn, Facebook and Twitter almost 2 years ago. I started with LinkedIn and almost a year later with my blog, recently i started using Twitter and Facebook. I also tried Plaxo and Hyves but I dropped them.

My first reaction to Twitter and Blogger was the same as the one I get today:

I have no time to Blog
I do not know what to Blog
Why should I Blog?

And with Twitter:

I do not know what to Tweet
I am not interested in what a person does in the bathroom
Too may Tweets to read
Stupid stuff

Within this blog item i will try to explain why I think it is a necessity (or can be) for every consultant.

Blogging

I started blogging with knowledge storage in mind. When I found a solution for a particular problem or when I did not want to forget how I did some things, i started to make a Blog item for it. This way I could always look up how to do this. After a couple of Blogs i got some remarks/comments/questions on my Blog, so i started to realize that my blog was also good for personal branding.

I also noted that it was also good company branding, because at seminars and conferences, people started to "recognize" me, so it was very good for networking as well.

Twitter

With Twitter the same story. I just wanted to know what it could mean for me, so i started using it out of curiosity. I use it almost only for business related tweets. I started to follow a lot of interesting people about interesting topics within my work field. And they follow me back as well. I use the Tweets to promote my Blogging site. I noticed that a lot of people started to read my blog and i started to read a lot of other interesting blogs.

So for me Twitter:

Brings me in contact with interesting people
It extends my knowledge within the field
It gives me personal exposure within the field
It gives me company exposure within the field
It gives me interesting insights in people's mind and thoughts within the field
It keeps me up-to-date with the latest insights within the field of EAI/SOA/BPM/Cloud
It extends my virtual network of people within the field

What to Tweet?

Well Tweet about what keeps you busy as a consultant
Tweet about topics that you would read yourself
Interesting articles, blogs and websites
Tweet about a new Blog item you made
Tweet about an interesting project your company finished
Tweet about an interesting solution you advised for a customer

When you follow a lot of people you can use the Twitter's List option. This way you can add people to a particular list, for example BPM, EAI, SOA, Oracle etc.

Who to Follow?

I follow interesting people within my field but also people with other interesting topics, like social media.

I also people follow people that have some interesting insights of what's going on at a customer. This can help me as a consultant what problems or thoughts customers have.

LinkedIn

I use LinkedIn as my personal CV and keep it up-to-date. This is a site that customers always will check and it also contains a link to my blog- and twitter site. It will give me leads to interesting projects.

Conclusion

I think that social media is a common good within the Consultancy profession. It will keep you up-to-date of the latest insights within the fields and you can track customers need.

Oracle BPM Suite 11g in Pictures

2011-02-08T14:57:00.002+01:00

Overview

Licencing

Modeling tools and Users

Business/Enterprise Architect

Business Strategy
Implement Business Architecture
Process Analysis/documentation

Business User

Understands the business
High level business processes

Process Analyst

BPMN 2.0 expert (business view)
Understands the details of the business
Make implementation decisions

Process Developer

BPMN 2.0 expert (technical view)
WebServices, XML
Enterprise Apps
Implement processes and UIs

Collaborative Design

BPM Studio

Process Analytics

Social BPM

SOA Platform

End-to-End instance and Flow checking due to integrated platform.

BPMN 2.0 support

Change with BPMN 1.1 :

Model execution
Models are interchangeable
XPDL and BPEL were used to interchange and execute

Interactive task and workflow

BPM Studio supports Simulation.

BPM Studio Business Indicators

Snapshots of business indicators must be added to the workflow and this is a little awkward because it blurs the process model.

Good integration with BAM

Process Composer

Within BPM Studio you design templates and there you set the constraints which changes can be made within Process Composer.

Unstructured Processes

End Users can

Reassign, reroute, delegate tasks
Add users for a task

Business Rules

Decision tables
If-then rules

Can be changed in Process Composer.

Participants lists can be dynamically assigned using a Rule.

Enterprise Manager

Unified Security

Clustering

SOASchool Certification: Module 8

2011-01-14T15:21:00.004+01:00

Today I had my SOASchool.com Module 8 exam for the SOA Certified Architect certification.

Suggested preparation:

You have to follow a course or buy the selfstudy kit
Read the chapters of the SOA Design Patterns book
Read the reader of the selfstudy kit
You really need to understand the Patterns and also the use of the patterns in relation to each other

Questions

The questions are the same as the other exams
Select All That Apply are the difficult ones and there were quite a lot

Next is the final Module 9 for me, and this is the Lab.

The Certification is valuable, because it shows that you know and understand the terms and challenges you face when working

within a Service Oriented Architecture. Or trying to work towards a SOA.

The nice thing about this certification is that it is Vendor Neutral. It does not mention a particular platform. It does however address

the SOA standards out there and that is a good thing of course.

The customers will also recognize this kind of certifications.

Oracle BPM Suite 11g overview

2011-01-13T16:12:00.002+01:00

Introduction

This blog item gives a short overview of the Oracle BPM Suite 11g.

If you as an architect, developer or sales person do no want to read all documentation, this can be a good starting point :-)

More thorough documentation can be found here.

Component Overview

The BPM Suite contains the following components.

As can be seen, the suite is layered on the Oracle SOA Suite and they share the following components

Business Rules Engine
Human Workflow Engine
Adapter Framework for integration

Oracle BPM Studio

Tool within JDeveloper to create and simulate business models using BPMN 2.0.

Oracle Business Process Composer

Web-base application that enables business users to collaborate with developers and designers.

A catalog of services, tasks and rules can be created in Studio and this catalog can be included in project templates, which can then be used to create new projects within

Business Process Composer. Here Business Rules can be edited at run time.

Oracle Metadata Service (MDS)

MDS Repository used to store information about deployed applications, used to share projects and project templates.

Oracle BPM projects are deployed as SOA Composite applications on the BPM Engine.

Oracle BPM Engine

It runs BPMN and BPEL processes, which are separate components but they share the same process core (i.e. security, audit trails, invoke services, persistence).

Oracle Human Workflow

This component contains functionality for:

Task routing to users, groups or application roles
Management of deadlines,escalations and notifications related to a task
Task Forms (including workspace and portals)
Organization,filtering, prioritization,dispatching rules

Oracle ADF can be used to develop custom Task Forms.

Oracle Business Rules

This component lets you automate policies, computations and reasoning in an agile way.

Of course it is the trick when to use this component to get that agility.

Oracle Enterprise Manager

This web-based application is used to control and manager applications running on the SOA Suite.

Process Participants applications

Oracle BPM Workspace and Oracle Process Spaces are used by process participants.

Workspace (old BEA) gives you the opportunity to:

View running process instances
Work/View task lists
Out-of-the-box dashboards
Custom made dashboards
Manage organizations and roles

Process Spaces is a collaborative workspace that gives you some Social Media, built on top of Web Center Spaces.

Oracle ADF and Fusion Middleware

2011-01-12T14:11:00.002+01:00

Today I browsed through some nice sessions on the Oracle ADF Framework.

I heard from the framework but not know what I can expect from the Framework and how it is related to SOA and Fusion Middleware.

The Framework is actually quite big and implements the Model-View-Controller architectural pattern which makes GUI development easier to develop and maintain.

Oracle sets it as "next generation" Web User Interfaces for

Fusion Applications
Middleware components like Enterprise Manager
WebCenter Spaces

The next picture shows an overview of the components of ADF.

So I asked myself how is this Framework to be positioned with Oracle SOA Suite, Oracle BPM Suite and Oracle Service Bus?

The next picture shows its place.

In my view within a SOA 5-layering architecture:

ADF can perfectly be used within the development of the presentation layer.
The OSB is generally used as an abstraction between all 5 layers. This means that all Services (Task Services, Entity Services and Utility Services) are exposed through OSB

But as I said, I am not an ADF expert, so ADF experts out there, how do you place ADF within your SOA Architecture?

Oracle working on BPM specialization

2011-01-12T12:20:00.002+01:00

Just like the SOA Suite specialization path, Oracle is working on a BPM specialization path for certification.

Note: For the links used in this Blog item you need to have an Oracle Partner Account.

The following information was already made available for Partners:

BPM knowledge zone, in which you can become a member of Oracle Mix which is a Social Network to share ideas and to ask questions.
You can become a member of Oracle BPM Suite and Oracle BPA Suite.

There is also a BPM Bootcamp, just like the SOA Suite 11g Bootcamp. This is a 4-day bootcamp and as a prerequisite follow 4 webcasts.
BPM Sales Guidebook which can be used by the sales persons and contains some management summaries.
The implementation Certification is not ready yet but there are some assessments available for PreSales Consultant and Sales.
You can test your knowledge after the BPM bootcamp here with an online assessment.

Integrating BPM and Social Media

2011-01-07T19:46:00.002+01:00

I find it interesting to note that there are a lot of Social Media Tools out there.

If you google on "social media tool" you get 2.650.000 hits.

When you do the same for "bpm tool" you get 14.600 hits.

You can say that there are of course a lot of small social media tools and that a BPM tool is a lot complexer.

Would it be wrong to ask myself if we could help business with (mostly free) social media tools instead of a (mostly more expensive) BPM tools?

Some arguments in favor of social media tools, i can think of:

Mostly for free
There are a lot
Mostly simple
Fun to work with

There are also some disadvantages i can think of:

How long will the tool exist?
How secure is it?
Can it handle a lot of users?
Can it handle the performance that is sometimes required?
What is the support?
Integration with other tools?

So probably the disadvantages will block CIOs from using them in business critical business processes, but maybe they can be very valuable in using for other business processes.

Some arguments in favor of BPM tools:

More secure because used on-premise
More support and documentation
More auditing information out-of-the-box
BAM
Better integration support with other systems

Some disadvantages:

Expensive
Complex

But how about a more Hybrid solution in which BPM and Social Media tools are used together or even integrated.

You already see this within the Cordys stack (integration with Google) and Oracle (uses collaboration tools).

I think we will see a lot more from this in 2011.

What are your thoughts on this?

Oracle Exalogic: An introduction

2011-01-05T13:44:00.002+01:00

Introduction

Oracle has introduced the Oracle Exalogic Elastic Cloud system, but what is it and what can you do with it?

This blog item is a small excerpt of the full article that can be found here.

The management summary

The Exalogic is: "the first system specifically designed to provide enterprises with a foundation for secure, mission-critical private cloud capable of
virtually unlimited scale, unbeatable performance, and previously unimagined management simplicity".
In other words the first private cloud supermachine and datacenter of the 21rst century.
It can run all kinds of applications of Linux and Solaris and is optimized for Enterprise Java, Oracle Fusion Middleware and Oracle's Fusion Applications.

Overview

The next figure shows the hard- and software stack.

Hardware

The InfiniBand I/O forms the hart of the horizontal scaling capabilities of the system. Fast connection to Exadata Database Machines.
There is a small, medium or full rack configuration with respectively:

8, 16, 30compute nodes and 96, 192, 360 2.93 GHz XEON Cores.

Software

Java and the entire Fusion Middleware stack is designed for deployment on Exalogic, also the Fusion Apps will be deployable on Exalogic.

Exalogic is compatible with all standard Oracle Linux 5 and Solaris 11 applications.

The Oracle Enterprise Manager provides full management through Grid Control and OpsCenter in which every individual hardware component can be managed in real time.

There have been special performance optimizations made within the Exalogic for WLS, Coherence and JRockit and Hotspot.

Exalogic can be directly integrated with Exadata using InfiniBand, the JDBC driver and WLS data source infrastructure can communicate natively using protocols available on InfiniBand.

Also the Active GridLink for RAC within WLS will improve the performance of OLTP applications.

The WLS server also takes advantage of the underlying hardware in its thread pooling algorithm for optimal Java application performance.

Conclusion

Exalogic is a high end, all in one "datacenter" for high performance and scalable private cloud specifically for Fusion Middleware, Fusion Apps and Oracle RAC.

It will probably also create a vendor lockin because the whole infrastructure and Fusion software is optimized for this system.

But for medium to large organizations this may be a perfect solution for their enterprise architecture and the management of it.

Oracle in 2011

2011-01-04T20:22:00.001+01:00

Introduction

This blog item highlights some points of Oracle for 2011 that will be be important for 2011.

Cloud in a box

As Larry Ellison stressed out:

The new competitive battleground in corporate IT will be centered on optimizing the performance of hardware-plus-software systems that have been specifically engineered for superior performance, speed of installation, and minimum of integration.

He wants to fight IBM for the #1 position on the high end computing servers. Its about the Exadata, Exalogic, and Sparc Superclusters machines.

For me as a Fusion Middleware guy I am curious in the Exalogic proposition and when I can first use it and develop on it here in the Netherlands.

Fusion Middleware Suite

Oracle has made a great step forward with the Middleware Fusion 11g stack, which is better integrated. The new 12c (c=Cloud) is on its way too !

The product is more integrated with fault management framework and security policy framework. It also contains the Event Delivery Network, with which you can

define and use Business Events in a Publish-Subscribe model.

Conclusion

Oracle will target at a full delivery from hardware and software, in the public- or private Cloud, on heavy machines and fully integrated.

I am curious to see if the Oracle Application Integration Architecture will pay off in the Netherlands as well.

Oracle SOA Foundation Practioner exam

2010-12-29T12:00:00.003+01:00

The Oracle Middleware Fusion 11g stack has a certification path for SOA:
1Z0-451 Oracle SOA Foundation Practitioner.
When you pass this exam you may call yourself: Oracle Service Oriented Architecture Infrastructure Implementation Certified Expert.

Today I passed the exam with 81% but I must say the questions were pretty detailed.
So you really must have practical experience with the Oracle SOA Suite 11g, but this is a good thing because that is where Certification is for.

Suggested preparation:
* Follow a SOA Suite 11g Bootcamp (or follow the book: Getting Started with Oracle SOA Suite 11gR1)
* Play with the product at home
* Follow the recommended learning path as mentioned here (for this you must be a registered Partner)

The questions are of high level and detailed level, with Choose Applicable, True/False, and Select Correct type of questions.

A CIBER colleague of mine also passed this exam: Michel Christianen.

Do not use generated database webservices in Cordys

2010-11-28T20:43:00.002+01:00

Introduction

Cordys BOP4 has the great option (just like all other respected EAI tool) to generate webservices on top of your databases.
In fact you can even generate a GUI on top of it, so that you can adapt data within your database through a GUI.
Sounds great or not !?

Watch out, because there can be a catch and this all has to do with a good SOA design!

Using webservices

In a project I work in, there has been decided to use generated webservices within the BPM processes.
This can easily be done by generating the MetaData of the database and then generate webservices for it, or even customized webservices by
using you own SQL statements.
This all works fine, but ..
1) You have to catch exceptions within all the BPM processes that use these webservices
2) The data within the databases can change (types, extra data elements, data elements removed, partitioning of data, primary key changes)
3) This will impact all the BPM processes that use these generated webservices

Define Data Services

The advise I want to give is to encapsulate the generated database webservices in Data Services (sub processes).
This way:
* Define the interface of the Data Service within a WSDL and use Contract By Design
* Call the generated database webservices within the Data Service
* Handle all technical errors within the Data Service, or expose when needed
* Use this Data Service within the other BPM processes

This way the changes of the database are encapsulated within the Data Service.

Conclusion

By using Service Oriented Principles within the Cordys BOP4 product it will increase the maintainability and usability.

Principles:
* Abstraction (by using Contract By Design pattern and Data Service)
* Loose Coupling (by adding just an extra layer on top of your generated database services, so that database changes can be handled better)

So actually use the generate webservices feature of Cordys !
But use it with care and keep the SOA principles in mind!

Cordys: Sharing Message Maps

2010-10-28T10:26:00.002+02:00

Cordys has a handy feature and that is of sharing Message Maps. A main process can share its message map with child processes.

The way to do this:

1) Select the sub process within your main process and select Properties

2) Check the Share Message Map

The Message Map is automatically copied to the subprocess. However sometimes this is not the case and the easiest way to do this is to copy
the message.

3) Goto the Mesage Map of the main process and select a Message and right click Show XML

4) Copy the XML
5) Goto subprocess and select Paste XML as Element

This feature is handy when you want to use a generic Message within your process

Exception handling in Cordys BPM

2010-10-22T11:32:00.004+02:00

A couple of days ago I was discussing the handling of all kinds of errors within your BPM.
In this case within the Cordys BOP4 product.
To handle technical errors (like a webservice can not be reached due to system crash) and functional errors the "normal" BPM flow gets blurred with all these exceptions.

We discussed ways to handle this:
1) Using different Views on your BPM (this is not (yet?) a feature in Cordys), so viewing your BPM with/without error handling)
2) Using different colors in your BPM
3) Using Embedded Subprocess

Example of option 2

Example of option 3

Cordys BOP External Services Configurator

2010-10-21T14:15:00.002+02:00

Introduction

For external webservices within the UDDI connector you have the possibility to configure the endpoints of the webservice.
This way in each environment (Development, Test, Acceptance, Production) you have a way to configure the endpoint differently.
You can use the External Service Configuration. However when I did this i ran into a problem.

Import WSDL

To use an external webservice you can import the WSDL.

Goto Workspace Folder where you want to create the webservice (probably a Web Services Folder somewhere).
Right-click New > Other > Web Service
Select Import WSDL as the source, give it a name and description
Next you have to give up the URL where the WSDL is located
Note: This can be ackward but for the moment not the issue i got

Give logical endpoint to the service

When you look at the properties of the imported webservice by double click on it you will see:

I advise to change the <serviceuri> into a more logical endpoint name instead of the one filled in after importing the WSDL.
This way you will be aware that the correct endpoint still has to be filled in within the External Services Configuration tool.

Goto External Services Configuration
There you will see the logical service uri and the physical endpoint

Problem when test webservice

However when i tested the webservice I got the following error:

So it appears that the logical endpoint is taken instead of the Service URL as defined within the External Services Configuration.

Solution

When you restart the UDDI connector it takes the correct endpoint.

Conclusions

Use logical names within the webservice properties
Define the real endpoint with External Services Configuration
Restart UDDI connector when the endpoint is changed

What to put in BPM?

2010-10-12T20:04:00.003+02:00

Introduction

When implementing a BPM or using a BPM tool I see a lot of functionality that (in my opinion) does not belong into that layer, but within a SOA Service.
But are there some general rules or guidelines that you use to decide what to put in BPM and what to put in a Service?
This blog item is a first try.

Put in a Service

This is a list of items/functionality that I would put in a Service:

Static calculations not dependant on business data (example: Fahrenheit to Celcius calculations)
High performance / complex calculations on data (example: complex calculations over series of data)
Technical orchestrations (example: Getting data out of several different databases)
Sending messages (in a particular format) to another department or even another organization (B2B)
(example: ebXML messages, HL7 messages)
Adapters (example: database-, file-, mailadapters)
These are always implementation specific in most cases and the implementation should be abstracted with a service.
However in most cases you will see these adapters used within the BPM tools.

Put in a BPM

This is a list items/functionality that I would put in a BPM:

A step in a business process implemented in a Service operation or sub process.
Scheduled triggers to execute a business process (example: every day execute a particular business process)
You can put the schedule in a business rule to make it flexible
Steps / Activities that mean something to the business
Remark: Discuss in depth because even business people dive into technical details.

You are invited to increase the list !

Conclusion

I think there will remain discussions where to put which functionality. With the BPM tools nowadays you can use integration software that you do not want in the BPM layer, like
a database adapter or file adapter. This should be put in a Service and abstracted. However time-to-market and ease of implementation is often a (maybe valid) argument to
do it like this. But hope you take a step back and think about it when you use a BPM tool because also with this tool you can make spaghetti !
And last but not least: Do not use a BPM tool as just another visual programming tool

SOASchool: Module3

2010-10-03T15:16:00.002+02:00

Friday I had my 3rd exam SOA Design & Architecture.
In this exam you also get 50 questions and you need to have 80% right.
This exam is about understanding the 8 principles of Service Orientation.
It has strong resemblance with the first module, but this exam was more detailed.

Up to module 8 of my SOA Architect Certification.

Service Orientation: Exception handling pattern

2010-09-27T10:16:00.002+02:00

Introduction

I am a great fan of the service orientation principles described by Thomas Erl: http://www.soaprinciples.com/
And also of its patterns: http://www.soapatterns.org/
This will not be a sucking up story about Thomas, but the need for a pattern on error handling.

Layering

One of the patterns is that of Service Layering, in which the Services can be categorized in the following layers:

The Utility Service Layer consists of Services that are of agnostic functional nature and are not functional related to a business entity or business process.
The Entity Service Layer contains Services that are related to business entities and are usually also agnostic.
The Task Service Layer is more related to Services that are specific to a particular task or process.
Usually you also have a Orchestrated Task Service layer contains Services that orchestrates Services to fulfill a business process.

My question is how to deal with errors in the separate layers?

Error Types

Let me first describe the type of errors we can have:

Technical Errors
Those are errors of technical nature, like database can not be reached, wrong message format. Mostly internal service errors that occur.
Capability Errors
Those are the errors due to the fact that a capability of a service can not be executed. These errors are more of a functional nature.
Business Process Errors
Those are errors that occur within the logic of a business process.

The next item you have to think about, is which errors you want to expose.

Expose errors?

The first option that comes to mind is that all errors remain internal, using the Abstraction Principle.
However if you think deeper then:

Technical errors may kick off other processes that will deal with service errors, by sending mails to maintenance for example. Then technical errors are part of the service contract.
A composite controller may handle a capability error by trying it again or using an alternative path.
A service does (generally) does not know in which activity it participates, so the service can not decide what to do when something happens. Then the error is also part of the service contract.

How to expose errors?

When you decided to expose errors then you have to think about how to expose the errors.
When looking at webservice services you have several options:

Return an error within the output message
Generate a soap fault
Generate a business event that something happend

I am more in favorite of the last two, because the first option always blurs your goodweather logic and this way the maintenance of your code can be more complex.

Conclusion

Hope this gives you some stuff to think about when designing your service contract.
Other suggestions are welcome !

Cordys: Connectors open source

2010-09-20T09:07:00.001+02:00

Taken from Cordys Wiki:

Cordys has always had two types of connectors. The first group of connectors are developed and supported by the Cordys platform engineers. Examples include the E-mail and the FTP connector. The second group of the connectors are developed and supported by the former Cordys professional services group (a.k.a. COE). Examples include the JMS connector and Command Connector.

Some time ago the professional services group moved to CSC, leaving the status of the professional services connectors a bit unclear. People were still maintaining professional services connectors, but it became more difficult because they needed to focus on CSC projects. Not on maintaining the connectors.

Another development is the creation of alternatives for platform connectors. The development of platform alternative takes a lot of time and blocks the possibility of including the enhancements into the product since they were on a different code base. So Cordys had to revise its strategy. One particular idea that kept showing up, both within as outside of Cordys: make the platform and professional services connectors open source. External developers will then be able to customize and enhance a connector instead of rebuilding it from scratch and Cordys will have the option to include a snapshot of the open source connector into the platform. Connectors shipped with Cordys will have support, which makes the customers happy.

The last two weeks this idea was validated with customers, partners and colleagues. All parties were positive about the overall idea. Thus Cordys decided to go forward with it.

The next step is creating a concrete proposal which includes details like timing, a definition of what Cordys support means, the location sources and a list of connectors to start with.

This is a good idea and will hopefully increase the number of connectors and also the quality. Looking forward to it.