SOA Evolution

ITIL Services

noreply@blogger.com (Merkel Marmaduke) — Sun, 03 Jan 2010 04:21:00 +0000

ITIL Services are different from Services Oriented Architecture (SOA) Services. The Information Technology Infrastructure Library (ITIL) is an accepted standard for operational IT issues such as incident management, problem management and change management. ITIL builds up a large set of terms and tries to use these terms consistently throughout its discussion of Service Management.

The conflict over the term "service" can lead to some confusing conversations between developers and operations staff. I have posted earlier on the definition of services in SOA. The ITIL definition is

"A service is a means of delivering value to customers by facilitating outcomes
customers want to achieve without the ownership of specific cost and risks. "
ITSMWatch

Two very different examples of a service in ITIL could be a payroll system and the resetting of a LAN password. The services of an IT organization should be available on a "service catalogue" and the clients of the IT organization can use this catalogue to choose and understand the services they receive.

This is different from an SOA service. An SOA service is a software component which:
• Hides its implementation,
• Is based on standards,
• Is location transparent,
• Is message coupled, and
• Is accessible through defined platform independent interfaces.

The main difference is that an SOA service is a software component in SOA. It is not the service that a help desk provides when they reset your password and it is generally smaller than a whole payroll system. An SOA service can be hidden away from the users in the cloud of application components. Ideally it is granular enough to have meaning to users but a user does not need to be able to interface with a service directly.

There can be coincidental overlap between an SOA service and an ITIL service. A service for paying a bill may also be presented directly to the users of an IT Department or organization and therefore be a service on their catalogue. This only seems to confound the issue. It is safer to assume that ITIL stalwarts and SOA engineers are talking about different things when they refer to a service.

IT Service Management (ITSM) under ITIL however can provide an appropriate means for providing governance for your SOA and for providing the operational platform for your SOA. SOA may contain more component parts than traditional systems and a disciplined approach to configuring, operating and changing these parts is required which is exactly what ITIL offers.

Interestingly ITIL also seems to redefine SOA as "Service Offerings and Agreements" and "Service Outage Analysis" which also makes life complicated.

Notes:
Some good articles on ITIL and SOA follow:
http://www.ca.com/Files/WhitePapers/soa_wp_jlong_33146.pdf
http://www.ibm.com/developerworks/ibm/library/ar-soaitil/
http://www.ebizq.net/blogs/bethgb/2009/03/can_itil_help_fix_soa.php
http://www.biske.com/blog/?p=467

The book I am reading which proffered the payroll as a service example is:

Klosterboer, Larry, 'Implementing ITIL Change and Release Management', IBM Press, 01 Dec 2008.

Oracle Redefines Middleware

noreply@blogger.com (Merkel Marmaduke) — Fri, 01 Jan 2010 04:30:00 +0000

Oracle Fusion Middleware 11g is the first real attempt to integrate the Oracle Fusion suite with the recent BEA products it acquired in 2008. According to Oracle this "Completes the integration of BEA into Oracle".

Oracle seems to be redefining "middleware" to include such things as Business Intelligence, Content Management and Collaboration which has typically been considered user-facing software. At the other end of the software stack Oracle looks at infrastructure services such as virtualization utilities.

"We take a broad view of what is included in Middleware" Rick Schultz of Oracle says on his podcast. According to the materials that Oracle make available, Middleware seems to be defined as a "Set of components that all applications need". This includes

User Interaction (Web 2.0, RIA, Mobile, Search)
Enterprise Performance Management (Planning, Budgeting, Scorecards)
Business Intelligence
Content Management
SOA and Process Management
Application Server
Grid Infrastructure
Development Tools
Enterprise Management
Identity Management

Depending on the spin, redefining middleware could be a good or a bad thing. I always felt I understood what middleware was. I agree with Wikipedia that "Middleware is computer software that connects software components or applications." To put it more plainly it is the glue that holds together different parts of a computer application and helps integrate computer applications.

The Oracle list is huge, and I suggest Oracle needs to come up with another name for all this software infrastructure (like Software Infrastructure Suite) which does not suggest it is simply the glue that holds the application together. User facing software infrastructure such as Web 2.0 collaboration facilities, Search, budgeting software, score card software and business intelligence is much more user facing than most custom built software. Other products on the list are part of the operating environment. This is infrastructure that the software developer should have little concern with.

This is not to say that the Oracle Suite is a bad idea. We have to accept that with SOA there is far more of an application that can delivered out of the box rather than written anew by developers. True middleware such as the ESB, is still a significant part of the Oracle Suite but the other components also need to be part of the solution architecture. Oracle focuses on some important aspects from the traditional middleware space including:
• Support for SCA (SOA Component Architecture).
• Support for Event Driven Architecture.

Oracle now has a concerted push to explain its offering to the world. This has resulted in seminars and workshops around the world. I attended a couple and they are worth the time to get an idea about what Oracle is proposing. I wish to thank the presenter for the descriptive term "Marketecture" (aka Marchitecture). This term seemed to gain popularity through Dilbert cartoons in 2009, after having some more serious uses back in 2003. Marketecture is the IT architecture as presented by marketing people and therefore has limited detail and may bear little resemblance to the technical architecture.

In conclusion, there seems to be much goodness in the new Oracle Suite. SOA can benefit from a large number of products to support the approach. Oracle Suite is much more than middleware, but Oracle should not try to cover everything under the "middleware" term just because its long list of acquisitions allows it to do so.

Other references:

http://www.dilbert.com/strips/comic/2009-03-02/, Marketect cartoon
http://www.uml.org.cn/success/pdf/marketecture.pdf, Marketecture vs Tarchitecture
http://www.it-director.com/technology/content.php?cid=6173, Architecture Marchitecture (spelt differently but same idea)
http://www.oracle.com/technology/products/middleware/index.html, The list of products in the Oracle Middleware stack
http://www.oracle.com/products/middleware/docs/microsite09-flashmedia-pdfs/fmw11g-overview-solution-brief.pdf, Overview of Oracle Middleware

SOA Definition Revisited

noreply@blogger.com (Merkel Marmaduke) — Sun, 14 Jun 2009 04:26:00 +0000

It is time to revisit the definition of SOA. In an earlier posting I provided the following definition and suggested it may evolve.

SOA is a software architecture where the business logic is encapsulated in services and a single application may be distributed over many processors.

A service in this context is a software component which·
Hides its implementation,
Is based on standards,
Is location transparent,
Is message coupled, and
Is accessible through defined platform independent
interfaces.

I recently came across this definition in a book published by IBM Press (see reference below).

A service-oriented architecture is a framework for integrating business processes and supporting IT infrastructure as secure, standardized components—services—that can be reused and combined to address changing business priorities.

Beiberstein et al refer to SOA as a framework I refer to it as an architecture which should be self evident. I do not think this is an important difference. The difference suggests another layer of abstraction a littler further away from the implemented product. The Zachman framework is even more abstract though.

There is an IEEE definition for architecture it is "the fundamental organization of a [business] system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution". I think this sufficiently describes what is needed for an SOA without out going to the next level of calling it framework. I will stick with calling SOA an architecture for now, although I acknowledge that there are a number of technical, logical, and business centric models that should be documented for an SOA.

I like the reference to business processes and I compare it to my discussion of business logic although business logic is at finer grain and is within a service. There is an interesting argument that Business Process Modelling (BPM) should be separated from SOA but generally a good SOA implementation will allow an organization to implement its business process more easily than with other architectures. The BPM tool may be considered to be outside of the services but still be part of the overall SOA. I'll watch this space and consider where I draw my SOA boundary.

Standardize components (services) are obviously central to SOA. I don't think they are necessary secure components. Unsecure services can still be SOA even though it may be bad SOA and security has been a bit of a challenge for the evolution of SOA.

Reuse and combination are important outcomes of SOA but they do not necessarily define SOA.

My SOA definition above is still looking strong. It has the possible deficiency that it uses the term "Message Coupled" rather than "Loosely Coupled". "Message Coupled" is not in wide usage. While "Loosely Coupled" is in wide usage and is descriptive to a point, I found it too loose in my earlier posting and still stand by it.

References
Norbert Bieberstein; Robert G. Laird; Dr. Keith Jones; Tilak Mitra, Executing SOA: A Practical Guide for the Service-Oriented Architect, IBM Press: The developerWorks® Series, May 2008, Page 4.

The definition of Architecture can be found in IEEE Std 1471-2000 Systems and Software Engineering—Recommended Practice for Architectural Description of Software-intensive Systems [2007-07-15]

SWOT on BPM

noreply@blogger.com (Merkel Marmaduke) — Tue, 09 Jun 2009 11:58:00 +0000

Toward the end of a problematic first implementation of a BPM solution, I was asked to evaluate whether it was worth pursuing this for future solutions in my organisation. The result was a SWOT (Strengths Weaknesses Opportunities and Threats) Analysis in which I put the case for persisting with BPM. An edited version of this paper follows. While I echo the optimism of vendors for large organisations there some cautions and costs that need to taken into account when embarking on the journey to adopt BPM.

Background
Business Process Modelling (BPM) is a technique for implementing IT systems that makes the process explicit usually through the use of a graphical tool that allows the process to be illustrated as a flowchart. This flowchart can be implemented directly as system of integrated application and services once these application and services have been built.

The process model represents processes of an enterprise, so that the current process may be analysed and improved in future. The development of a business process model is typically performed by business analysts and managers who are seeking to improve process efficiency and quality. The process is implemented by IT staff who may be assisted by BPM tools.

In theory, BPM tools provide business users with the ability to model their business processes, implement and execute those models, and refine the models based on real data. As a result, business process modelling tools can provide transparency into business processes, as well as the centralization of corporate business process models and execution metrics. In practice, BPM tools require significant training and needs users to work closely with other specialist IT staff to implement systems using these tools.

Our organisation has acquired Aqualogic BPM which is being used to redevelop the XYZ system. Vendor X are doing the development work. Aqualogic is new to both Vendor X and our organisation. This has meant both organisations have had to undertake learning in this technology. The project has taken longer than expected. The project will cost more than expected and the support model for this technology has not yet been developed.

This paper will assess the risks involved in moving forward with BPM and advise whether we should continue with XYZ system using the BPM product.

Analysis
When looking at the benefits related to BPM a distinction needs to be made between the BPM process and the BPM tools. Our use of BPM tools so far, has been to reverse engineer the XYZ system and implement an existing process. This is not the most beneficial way to use BPM and therefore will not immediately produce any savings for XYZ system.

More benefits are likely to flow from the revision of the business processes that is inevitable with complicated processes. The mapping and refinement on business process is the same as any continuous improvement initiative.

An effective BPM implementation is much more top-down. BPM is more suited to projects that involve a significant Business Analysis effort. The analysis of the process should be conducted at the business level. The process is then refined and optimised and where possible automated.

Computer Applications are no longer written purely for discrete business units. Modern enterprise applications link different business units with ever more complex processes involving people from all over the organisation. The emerging standard approach for this is to break applications into services (Service Oriented Architect) and link these together using BPM tools. In this way BPM can support large integrated enterprise applications. Along with a portal product and an enterprise services bus, a BPM tool is a central component of the modern SOA development toolkit.

If the right skills and tools are in place, BPM promises to enable a more agile organisation. Conventionally, changing software to implement a changed business process involves editing complicated program code possibly spread over a number of different applications. With BPM the process is presented in a more intuitively obvious form and can be stored centrally rather than spread across different applications. This enables changes and therefore more business agility because of the ability to change these processes more rapidly at lower cost.

Strengths

BPM allows the process to be viewed and communicated to non-technical users
The BPM can be changed by staff who understand the BPM tool
The business process is easier to follow than the corresponding Java code.
Status can be tracked using the BPM Tool
Possibilities for process change can be investigated using the BPM diagrams.
Supports Services Oriented Architecture (SOA)
Standards-based approach to integrating diverse systems and data sources.
Engenders Continuous Process Improvement
Improves business agility

Weakness

Staff need to be trained to use the BPM Tool
The organisation needs to pay significant costs to license the BPM tool (including non-production versions)
There are more points of possible failure in production systems.
Production systems need to be supported which may will involve extra servers.
Business process changes normally require constant involvement of the user. (Collaborative development model). This may require more user resource but also a change in software development approach.
The default process management screens on the BPM Tool may not be appropriate for users. In the case of XYZ system it is costing more to implement custom interfaces.

Opportunities

We could use the models in the BPM in our Architecture repository and benefit other projects.
Other projects will become easier to develop using BPM once we have sufficient skills and a critical mass of services that can be used with BPM.

Threats (Risks)

The organisation may not have chosen the correct tool and this may be replaced by a competitors product in the future.
The Aqualogic Tool may not be supported by the vendor in the future
It is possible to map more complicated processes. There may be a tendency to get automated tools to do this rather than simplify the business process.
There is a performance overhead in running BPM applications. This may cause programs to perform slowly and impact users.
BPM tool support from any vendor is limited in regional locations.
If we write custom interfaces for users to manage BPM workflow then we have to make sure these interfaces are compatible with future versions of the tool.
Outsourcers who have forged a partnership with our organisation will be able to charge high rates to support its BPM tool.

Conclusion
The choice of BPM for XYZ System was sensible from the point of view that the system was automating complicated business processes involving different roles. The decision was not ideal because the project did not require a business analysis stage and an opportunity to refine the business process.

The XYZ system project will therefore suffer the learning curve costs of the BPM Tool. It should also reap some benefits later on for having a process that can easily be modified our organisation retains sufficient skills to do this.

The BPM version of XYZ system should still be implemented. Handover to support should involve making sure there is an adequate support model for the product. This should involve having someone in-house that can support the runtime component of the BPM Tool and assist with the design-time component with the tool.

The investment in in-house skills should be used in other projects. The identification of the potential to use BPM should made early in a project so that the Business Analysis can be done with BPM in mind.

The risks are significant but most can be addressed by acquiring the necessary skills to use the BPM tool effectively. The benefits are also significant. An effective BPM capability will assist build integrated systems effectively and efficiently, and support implementation of a Services Oriented Architecture.

References

Rudden, J, Making the Case for BPM: A Benefits Checklist, BPTrends Jan 2007, http://www.bptrends.com/publicationfiles/01-07-ART-MakingtheCaseforBPM-BenefitsChecklist-Rudden.pdf
Sinur, Jim, The Top Five Benefits That BPM Delivers Today, Feb 4 2009, http://blogs.gartner.com/jim_sinur/2009/02/04/the-top-five-benefits-that-bpm-delivers-today/
Khadye, Vinayek, Benefits of BPM Systems, 7 Mar 2005, http://it.toolbox.com/blogs/bpm-blog/bpm-series-benefits-of-bpm-systems-4837
Cooper, Mark and Patterson , Paul, Business Process Management (BPM) Definition and Solutions, May 2007, http://www.cio.com/article/106609/Business_Process_Management_BPM_Definition_and_Solutions?page=5

SOA Adoption for Dummies Such as Managers

noreply@blogger.com (Merkel Marmaduke) — Mon, 23 Mar 2009 09:35:00 +0000

Software AG has commissioned another SOA publication to "Dummies" series and released it to their customers. This follows on from their "BPM Basics for Dummies" book which was a referred to in an earlier post. I have to confess to listening to the podcast version of this book while watching the cricket rather than reading the book in detail the conventional way.

This book is focused on SOA Adoption rather than SOA. It is not a technical book in SOA. References to XML, SOAP and WSDL are consigned to a footnote. This book is evangelical and pitched at management. I question whether it is right to consider that dummies would be involved in SOA adoption. Your average person involved in an SOA Adoption decision is a highly credentialed senior IT manager on one hand but on the other hand the time taken over each decision of a manager can be minimal and display all the appearances of being made by a dummy.

The authors should be congratulated for taking SOA Adoption as subject in itself as it is a difficult for enterprises to adopt SOA and it is much more than just the technical challenge that needs to be addressed. The book and podcast series are well produced and easy to absorb.

Do not expect any great depth from this book however. The authors call their SOA adoption approach "Rocket Science" but this approach simply consists of:

1. Keep the pointy end of the rocket up.

2. Keep moving up.

3. Don’t stop till you are weightless.

This is good motherhood advice for any significant undertaking but I do not see it as being particularly insightful for SOA adoption.

The authors do a good job of SOA evangelism and have lived up to the creation of simple book for "Dummies" by missing a lot of the technical detail. There is nothing particularly radical in this book but it does give you some important things to watch such as SOA Governance, Organization structure, SOA infrastructure, the SOA competency centre and organization structural issues that might arise from SOA.

In summary, this is a good book to give a dummy or manager who needs to be brought on board to get with your SOA program. It will not get you to SOA weightlessness without a lot of other expertise, skills and resources. It is not the bible for someone wishing to be the main driver behind SOA adoption.

Reference

Miko Matsumura, Bjoern Brauel and Jignesh Shah, SOA Adoption for Dummies , Wiley Publishing Inc., 2009
http://www.softwareag.com/Corporate/res/books/soa_adoption_for_dummies/default.asp

SOA and Cricket

noreply@blogger.com (Merkel Marmaduke) — Sat, 21 Mar 2009 05:32:00 +0000

It has been a long while since I put together a blog posting. This has been for a number of reasons both personal and professional. But one reason has been the game of cricket.

I have two sons that are good at cricket and play for a local club. Most of the English-speaking world knows what Cricket is. For the Americans who may read this it is a bit like baseball. You use a bat and a ball and not much happens for long periods of time. Unlike baseball it can be played over several days for up to six hours a day. Much of my time has been involved in watching the game played by my sons but also watching the televised games between the Australian national team and the South African national team.

Progress is made slowly in cricket and it is often not clear whether you are winning until near the end of the game. The players need to stay focused on what they are trying to achieve and there are a whole range specialists (batters, bowlers and fieldsman) involved. SOA is much the same. It takes time. Since my last blog I can report some real progress from my organization but we have still only completed a small part of the journey. We need to stay focused on the desired SOA outcomes and we are gradually building a set of skills and relationships that are required to bring SOA to fruition.

I intend to prepare more blog postings shortly. The cricket season is over and the hot spell in South Australia has passed. Meanwhile my organization has completed a number of projects and has embarked on more which will advance its SOA so I have some content to provide and hopefully the motivation to provide it.

Understanding Enterprise SOA: Book Review

noreply@blogger.com (Merkel Marmaduke) — Sat, 16 Aug 2008 04:56:00 +0000

I was handed this book by one of my programmers. He lent it to me because he knew I was doing some reading on the subject and trying to work out how to bring my software developers up to speed with SOA. I didn't get the impression that he had got terribly much from the book himself. After reading it I can see there is benefit for managers but I think there was not enough technical meat in the book for programmers and analysts.

This book is not technical. It can be read by managers, salesmen or students and I think these were the intended audience. The explanations are easy to understand although I found them long-winded and overly simplistic on many occasions.

The authors attempted to explain procedural code as if it was museum artifact. This made me feel very old. It begs the questions whether the developers brought up on object-oriented development and SOA really need a detailed explanation of the concept of procedural code.

On the positive side this books tackles issues that others do not. It has a good non-technical discussion about security issues and there is a lot of discussion about human issues such as training, reorganization and corporate culture.

Oddly for a book written in 2006 I did not find any mention of Enterprise Service Bus (ESB) although it was clear the author was encouraging looking at off-the-shelf SOA management solutions as well as SOA security and routing software.

Eric Pulier in his preface is unbridled in his enthusiasm for SOA standards:

In a spectacular cycle of enlightened self-interest, every major company in the world has build on top of standards which have ushered a potentially exponential release of value from investments in information technology.

Elsewhere in the book there are sufficient cautions to temper this enthusiasm. In particular the authors advise that much of SOA is still hard work and that there are no industry standard solutions to problems such as transaction processing and security.

There is a long and detailed case study running through the book. The authors do not hold back on the multitude of issues that confront an organization adoption SOA. Parts are realistic and even painful for those of us who have been through implementations of a difficult project or technology.

The book has a good history of technologies and standards. The authors make a good point about standards conforming to the Tipping Point model (see references below). This is when a standard reaches a certain level adoption that it suddenly becomes desirable for the majority of developers.

This is not a strong academic text. There are not many references to other information sources. The definitions are not clearly stated and often over-simplify the concepts. Concepts like Enterprise Architecture, Real-Time Computing and loose coupling are all not accurately defined.

"Loose-Coupling" is described but not adequately explained. The authors give the impression that loose-coupling is only about where you can locate services.

Web services, in contrast are loosely coupled. Once a piece of software has been exposed as a web service it is simple to move it to another computer.

This is an important aspect of coupling but is not the whole story as I have identified on previous postings.

The book focuses on synchronous services rather than asynchronous services. This is unfortunate as other texts such as Thomas Erl or Krafzig, Banke & Slama seem to advocate asynchronous services.

The book addresses the Subject of Enterprise Architecture again loosely defined. At one point the book states

There is no institute providing credentials for enterprise architects.

I'll give the authors the benefit of doubt. Perhaps in 2006 there were not the credentials around but these days there are a number of places to get recognized credentials as an IT Architect. The Microsoft Certified Architect programs spring to mind although I will not vouch for the content of these programs.

The authors see SOA as a method for replacing parts of the Data Warehouse. I was not particularly convinced by this. This was an interesting subject which I had not seen before. I think there is benefit in moving the content of SOA logging to the data warehouse but I was not convinced that SOA radically changes the typical data warehouse ETL process.

This is not a book with detail examples of SOAP and WSDL. It will not explain how to code or build services. You will not find much detail on WS-* here either.

Contracts and Service Level Agreements (SLAs) are described in the book. The authors also introduce the Business Level Agreement (BLA). This is agreement to monitor business conditions such as an inventory pile-up causing $100k per minute of cost.

The book is well illustrated with copious diagrams and notes attached to the diagrams. I found myself skipping past a lot of the diagrams because they did not seem to be explaining anything particularly special.

I congratulate the authors for including detail about setting up training programs in the book and explaining how the resources are brought in to deal with the adoption of SOA. This is missing from many other texts and is an important issue today.

The authors emphasize that SOA is an enterprise approach. Under SOA, IT departments put in place infrastructure to serve the enterprise rather than just serving particular applications.

I would recommend other books before this one for most audiences. The simplistic and at times pedantic prose would make it hard to recommend to an IT professional. Nevertheless there are some well-made observations in this book that you are not likely to find elsewhere.

References

Understanding Enterprise SOA, Eric Pulier and Hugh Taylor with forward by Paul Gaffney, Manning, 2006

The Tipping Point: How Little Things Can Make a Big Difference (ISBN 0-316-31696-2) is a book by Malcolm Gladwell, first published by Little Brown in 2000.

Will BPM be the Savior of SOA?

noreply@blogger.com (Merkel Marmaduke) — Sun, 03 Aug 2008 13:35:00 +0000

My friendly Software AG representative recently gave me a copy of the "BPM Basics for Dummies". This was an interesting little book pitched at the Executive level. It would not help an IT person to get started with any particular product set but would help them understand what all the fuss was about.

BPM (see note below) has been with us since at least the 1980s. It used to a paper process of writing up the process and then getting a team of programmers to build it, then implementing it on usually a big computer that could handle it and then have the computer operators monitoring it.

In the 1990s Business Process Reengineering (BPR) was an approach which changed an organizational process. It involved process diagrams, IT change and business change.

The excitement with BPM is that some of the leg work involved in actually building systems can be removed if the components of the business process are already established. In this case the components are Web Services enabled through an SOA.

What interests me about this is that some of the concepts behind BPM are a lot easier to explain to the business than SOA. You can get out the Lego blocks and explain the benefits of agility until you are blue in the face but you are still only talking in analogies and expecting the business to take a leap of faith. If you show the business a diagram of one of their business processes and then tell the business the cost of changing that business process then that makes sense in terms the business understands. Of course there is a significant investment in infrastructure, tools and services to establish before you get to that point. It is this significant investment that the IT executive wants to sell. If you are pushing a plan to buy infrastructure, tools and establish services then the plan will make more business sense if the objective is BPM rather than SOA.

I think Software AG understands this and that is why they are giving out "BPM Basics for Dummies" and not "SOA Basics for Dummies". I think the business will get behind BPM before they will embrace SOA. We may see more push for BPM to business executives and SOA may take a back seat. The IT department however knows it needs SOA to get BPM to work. That is just how it meets the business goal of BPM.

Note:

BPM stands for Business Process Management, but you would excused for thinking it stood for Business Process Modeling. The "Management" implies that we are doing a lot more than mapping out the processes. The "Management" moves beyond documenting the processes to making them happen and monitoring their effectiveness.

This is not to be confused with BPMN (Business Process Modeling Notation) where the "M" stands for "Modeling". BPMN however can move beyond an aesthetic model to an operational system by converting it to BPEL.

References:

http://www.ebizq.net/hot_topics/bpm/

http://www.bpmn.org/Documents/Mapping%20BPMN%20to%20BPEL%20Example.pdf

http://en.wikipedia.org/wiki/Business_process_reengineering

Kiran Garimella, Michael Lees and Bruce Williams, "BPM Basics for Dummies (Software AG Special Edition)", Wiley Publishing Inc. 2008.

Laws of Software Development

noreply@blogger.com (Merkel Marmaduke) — Mon, 21 Jul 2008 12:32:00 +0000

I ran across an new eponymous law while reading "Understanding Enterprise SOA" by Eric Pulier and Hugh Taylor.
Pulier's rule of reuse:

A programmer will look at another programmer's output, no matter how brilliant and declare it garbage

The best law's and rules are named by others who admire the author's work, but this one seemed pretty worthy to me. This got me reminiscing about some laws of software development that perhaps had the same rigour as Newton's Laws, Boyle's Law or Hooke's Law.

The rules I have admired during my career are:

Brooks's Law:

adding manpower to a late software project makes it later

Hofstadter's Law:

It always takes longer than you expect, even when you take Hofstadter's Law into account.

Codd's Rules of relational database management systems which I won't quote here because there are 13 of them.

Moore's Law:

The number of transistors that can be inexpensively placed on an integrated circuit is increasing exponentially, doubling approximately every two years.

This has a number of applications outside of transistors such as processing speed, memory and disk capacity.

Clarke's 3rd Law:

Any sufficiently advanced technology is indistinguishable from magic.

The reason I class this as a software development Law is that it points out the futility of trying to explain the technical details of software to Business users. If a technical decision does not have any business impact then it may as well be magic.

I thought I would check what other software development Laws there are on the net. Here is a good sample from some useful sources listed in the Bibliography below:

Amdahl’s Law

The speedup gained from running a program on a parallel computer is greatly limited by the fraction of that program that can’t be parallelized.

Asimov's laws (Yes its SF but one day a software developer might have to follow them):

A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey orders given to it by human beings, except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Atwood’s Law:

Any application that can be written in JavaScript, will eventually be written in JavaScript

Bye's First Law of Model Railroading

Anytime you wish to demonstrate something, the number of faults is proportional to the number of viewers.

Clarke's 1st Law:

When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

Clarke's 2nd Law:

The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

Clarke's 4th Law (He's not often credited with this one):

For every expert there is an equal and opposite expert.

Conway's Law:

Any piece of software reflects the organizational structure that produced it.

Edwards' law

You cannot apply a technological solution to a sociological problem.

Greenspun's Tenth Rule of Programming (There actually are no rules 1 to 9)

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp

This can also be worded “Those who do not understand Lisp are doomed to reinvent it.”

Gustafson's Law (also known as Gustafson-Barsis' law)

Any sufficiently large problem can be efficiently parallelized

Kerckhoffs' law on secure cryptography

A cryptosystem should be secure even if everything about the system, except the key, is public knowledge

Lehmann's Laws

Systems that are used must change or automatically become less useful
Through changes the structure of a system becomes ever more complex and more resources are needed to simplify it

Linus's law — named for Linus Torvalds,

given enough eyeballs, all bugs are shallow

Lubarsky's law of Cybernetic Entomology:

There is always one more bug.

McLuhan's Law:

If it works it's obsolete.

Metcalfe's law

In communications and network theory, states that the value of a system grows as approximately the square of the number of users of the system.

Murphy's Law

If anything can go wrong, it will.

Occam's Razor:
There are many wordings for Occam's razor and debate about how it is interpreted but from software development I think the best wording is "The simplest solution is usually the best". This is very nearly the same as the KISS principal "Keep it simple stupid".

Putt’s Law:

Technology is dominated by two types of people: those who understand what they do not manage, and those who manage what they do not understand.

Sturgeon's Revelation (sometimes referred to as his second law):

Ninety percent of everything is crap

Weinberg’s Law:

If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.

Wirth's law

Software gets slower faster than hardware gets faster.

Zawinski's law

Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.

Here are still more great laws from Joey DeVilla's Blog posting.

The Law	Who Said It	What it Says
Ellison’s Law of Cryptography and Usability	Carl Ellison	The user base for strong cryptography declines by half with every additional keystroke or mouse click required to make it work.
Ellison’s Law of Data	Larry Ellison	Once the business data have been centralized and integrated, the value of the database is greater than the sum of the preexisting parts.
Flon’s Axiom	Lawrence Flon	There does not now, nor will there ever, exist a programming language in which it is the least bit hard to write bad programs.
Gilder’s Law	George Gilder	Bandwidth grows at least three times faster than computer power.
Grosch’s Law	Herb Grosch	The cost of computing systems increases as the square root of the computational power of the systems.
Hartree’s Law	Douglas Hartree	Whatever the state of a project, the time a project-leader will estimate for completion is constant.
Heisenbug Uncertainty Principle (restated)	Jim Gray	Most production software bugs are soft: they go away when you look at them.
Hoare’s Law of Large Programs	C. A. R. Hoare	Inside every large problem is a small problem struggling to get out.
Jakob’s Law of the Internet User Experience	Jakob Nielsen	Users spend most of their time on other sites. This means that users prefer your site to work the same way as all the other sites they already know.
Joy’s Law	Bill Joy	smart(employees) = log(employees), or “No matter who you are, most of the smartest people work for someone else.”
Lister’s Law	Timothy Lister	People under time pressure don’t think faster.
Nathan’s First Law	Nathan Myhrvold	Software is a gas; it expands to fill its container.
Ninety-ninety Law	Tom Cargill	The first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time.
Pesticide Paradox	Bruce Beizer	Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual.
Reed’s Law	David P. Reed	The utility of large networks, particularly social networks, scales exponentially with the size of the network.
Sixty-sixty Rule	Robert Glass	Sixty percent of software’s dollar is spent on maintenance, and sixty percent of that maintenance is enhancement.
Spector’s Law	Lincoln Spector	The time it takes your favorite application to complete a given task doubles with each new revision.
Spafford’s Adoption Rule	George Spafford	For just about any technology, be it an operating system, application or network, when a sufficient level of adoption is reached, that technology then becomes a threat vector.

Some unattributed "laws" that are worth mentioning:

Build a system that even a fool can use, and only a fool would want to use it.
Any program over 100 instructions can be simplified by 3 instructions (without losing any functionality).
Any idiot can learn to use computers, and many do
There’s never time to do it right in the first place, but there’s always time to do it over when it doesn’t work.

This posting started with Pulier's Law: "A programmer will look at another programmer's output, no matter how brilliant and declare it garbage". At the risk of repeating Pulier's conceit of naming law about himself I propose:
Kimber's Corollary:

No amount of documentation is ever sufficient to completely understand a system

Bibliography
http://en.wikipedia.org/wiki/Adages_named_after_people

"Moore's Law is Dead", http://www.techworld.com/opsys/news/index.cfm?newsid=3477

http://en.wikipedia.org/wiki/Change_management_(engineering), This includes the reference to Lehmann's Laws siting Hinley, D.S. (1996). Software evolution management: a process-oriented perspective. Information and Software Technology, 38, 723-730

http://en.wikipedia.org/wiki/Moore

http://en.wikipedia.org/wiki/Occam

http://www.webopedia.com/TERM/C/Codds_Rules.html

Joey de Villa, http://globalnerdy.com/2007/07/18/laws-of-software-development/

Phil Haack, 19 Eponymous Laws Of Software Development, http://haacked.com/archive/2007/07/17/the-eponymous-laws-of-software-development.aspx

Eric Pulier and Hugh Taylor, "Understanding Enterprise SOA", Manning Publications, 2006

Book Review: J2EE and Weblogic

noreply@blogger.com (Merkel Marmaduke) — Sat, 19 Jul 2008 12:24:00 +0000

I was drawn to reading the book "J2EE Web Services on BEA Weblogic" by Anjali Anagol-Subbarao because my organization has purchased the Weblogic platform and I wanted more detail about how to make the best use of the tools available in this suite. General books on SOA can tell you about the principals and standards but without actually talking about specific tools there is a big gap between theory and practice.

"J2EE Web Services on BEA Weblogic" was published as part of the Hewlett-Packard Professional Book Series. While the Weblogic content the descriptions of some of the management functions of HP Openview were a bit alienating as I have not used this product. However it did also talk about some useful utilities and open source products that were not provided by either BEA or HP.

While the description of the tools were fairly high level and probably could be translated into similar functions in other suites I do not expect this book attract users of other SOA platforms suites.

The book style is fairly easy to read with some good diagrams and examples. The SOA standards and principles are covered well with a focus on the WS-*, SOAP and WSDL standards.

The basics of J2EE and EJBs are explained and some fundamentals of the Weblogic platform. This is not an evangelistic book about SOA or Web Services but goes into a bit about the technology of how to bring it about. The SOA benefits are there but the SOA story gets a bit lost amongst the instruction on how to use the BEA tools.

Some of the material reads like BEA marketing literature. I would have liked more practical tips on what to use and what not to use. In a book like this we need information that we can't glean from the manuals. This provided the high level overview with out some of the more practical "dos and don'ts".

There is good information on versioning and a good section on evaluating WS management tools. Also the book provides some practical approaches to performance issues.

This book has limited readership and will date quickly. Your best technical advice will always come from the product manuals but this book explains a bit about the suite is used as a whole. Published in 2004 this book is nearing its "use by" date but there is still some relevance to what it describes. Be quick though, with the acquisition of BEA by Oracle we can expect some changes to the BEA product suite.

Lego, Carrots and Coleslaw

noreply@blogger.com (Merkel Marmaduke) — Sun, 27 Apr 2008 02:48:00 +0000

I discovered Jeff Schneider's blog archive about SOA and Lego blocks. He has added some great photos. Unlike some other photos illustrating SOA on the web these blocks on the photo are real lego. Jeff discusses accessibility, recombination, composite applications and peer-oriented systems using the metaphor of lego blocks. He also makes judicial used of carrots, cabbage and coleslaw.

I have discussed some of the advantages and disadvantages of the lego metaphor in my earlier posting Lego and SOA. It has also made an appearance in some videos I have seen and conferences I have attended.

Web Services versus the REST

noreply@blogger.com (Merkel Marmaduke) — Sun, 27 Apr 2008 02:21:00 +0000

Despite there being much discussion about the relative strengths on between SOAP Web Services and REST it is not actually that often that you make an explicit decision between the two. Once the die is set you will tend use one or the other. Alternatively it may be self-evident which approach gets used for what services.

The decision to use one technique or another is not made very often. Often such a decision is made in series of smaller decisions over a period time. For instance REST may be chosen to be used for some aspect of some project several times over before it assumes the role of accepted practice in an organisation. Often these decisions are not made in a formal way.

This week my organisation had one of those smaller decision points. The solution architect proposed that we use REST or SOAP Web Services for our services. To define the solution architecture fully we needed to decide which one we going to use.

The case for REST was:

There would be less processing overhead
It would a less heavyweight solution as far as programming effort
Easy for service consumers to understand

The case for SOAP Web Services was:

The project was already using SOAP elsewhere
Web Services was a standard and REST was a "architectural style"
Our partners were more likely to adopt Web Services than REST
The explicit definition of service contracts in WSDL would be an advantage
Can deal in standard fashion with reliability and transaction issues
Better for modelling processes (verbs rather than just nouns)

Other points that were considered

Our organisation is not particularly experience with either SOAP or REST
We had used other "service" approaches (neither SOAP or REST in the past)

The decision is not a simply a decision between one or the other. An organisation can adopt neither, both for different circumstances or both in combination. Zapthink advises

… the debate about Web Services and REST is as pointless as arguing whether a
hammer or a screwdriver is a better tool.

W3C discuss a scheme for reconciling the two so that SOAP-based services can be used in RESTful ways.

The winner in this case discussed above was Web Services. This does not commit the organisation to using these technique in the future however this decision will influence future decisions to adopt Web Services in favour of REST. The "horses for courses" approach of choosing the best approach for the application will become less likely as the organisation developers expertise and confidence with Web Services.

Notes:
There is some argument about whether REST is Web Services or not. The use of Web Services Description Language (WSDL) in conjunction with SOAP and not REST suggests that SOAP and Web Services are inseparable. I will use Web Services (capital "W" and capital "S") to describe SOAP based services. Many commentators advocate using "web services" as a description of services developed in REST.

Some REST (Representational State Transfer) include

Daniel Rubio
http://www.webforefront.com/archives/2005/05/rest_-_represen.html

Joe Gregorio, December 01, 2004
http://www.xml.com/pub/a/2004/12/01/restful-web.html

Mark Baker et al
http://rest.blueoxen.net/cgi-bin/wiki.pl?FrontPage

Evolution: The Fittest and the Sexiest

noreply@blogger.com (Merkel Marmaduke) — Sun, 30 Mar 2008 06:44:00 +0000

The topic of this blog is "SOA Evolution" so it seems overdue for me to look at the Evolution process. I coined this title to chronicle my organisation's slow but deliberate adoption of SOA. 'Evolution' seemed to be an appropriate term because it was both a slow process but one that would improve our applications. In this posting I will look at evolution more literally and find more appropriateness in the term than I first imagined.

I am reading a book by Richard Dawkins (see references below) on a range of topics but primary about evolution in biology. There are two principal evolutionary processes. One is the survival of the fittest and the other is sexual selection which is the process of influencing the evolution process by the choice of a mate. The success of evolution is its ability to adapt to its environment and to succeed against other species in that environment.

Natural adaptations are gradual and based on the parent life form. The history of the life forms can be discerned in the DNA of the life form. We can determine that humans evolved from an ancestor we share with Chimpanzees from looking at this DNA.

The process of survival of the fittest can be applied to software development technologies. We see various versions of programming languages replaced by later more effective versions. We see whole languages disappear into obscurity (eg. Algol, APL, SmallTalk, ADA) or replaced by new languages. Relational databases took over from set-based or network databases. Software development approaches have evolved from structured programming, to object-oriented programming to SOA. It has been mentioned that SOA was forged from marriage of distributed computing and document-based integration.

Extinction of old approaches is often not as complete as we would like. The Cobol programs and the old database technologies are still around. The old dinosaurs are hard to kill off completely.

As with any evolutionary process the origins of software development are evident when we dig deep. Visual Basic .NET is still saddled with some of the terminology and constructs that were around in DOS Basic. The kernel of some operating systems still contains some anachronisms which would not be used if it were not for historical reasons. Many software systems have parts that were written long ago or copied from other systems that no-one can maintain or just do not make sense in the context of software written today. Some software maintenance tasks are like archaeological expeditions. Unfortunately it is not passive fossils that the software archaeologist finds amongst the code. Some of these historical curiosities actually support critical business functions.

There are some principles from long ago that we do not want to discard though. Just like some of the primitive instincts and emotions buried in the core of our brain there are principles from long ago that should remain in our software. Procedures and functions still have a role in our software as do objects. The development of services does not replace the need for these other means of encapsulation. The principles for strong typing and structured programming still have as much value today as they did thirty years ago.

The mechanism for natural selection process in software technologies is not that different from the animal kingdom. Programmers will rewrite software using new techniques if requirements change. New tools outsell older tools that do not provide the same benefits. New techniques that reduce development time or improve quality replace old techniques.

Just as species become extinct old software technique die from disuse or replacement. Old code does not die off as quickly as some of us would like. It can last forever in our systems if the requirements it fulfils and infrastructure it uses do not shift too markedly.

Charles Darwin had two theories that he espoused in his books "The Origin of the Species" and "The Descent of Man". One could be paraphrased as 'survival of the fittest' and the other once was 'sexual selection'. Sexual selection is where a mate is selected because he or she is bigger, is stronger, is darker, has more colourful tail feathers or has less hair and therefore individuals with these characteristics tend to propagate.

I have briefly explained how the fittest software and development techniques succeed at the cost of inferior versions but it is also true that the sexiest software can be propagated despite any objective superiority. This can happen through good marketing or the support of a major company. Software standards are often agreed upon, even though they may not be the best based on an objective measure of quality. Often the benefit to adopt a standard exceeds the cost of the standard being suboptimal.

'Survival of the fittest' and 'sexual selection' are alive and well in the software industry. I expect SOA to remain an important part for the genetic makeup of future software development even after the term has lost its currency.

References

Dawkins, Richard, A Devil's Chaplain: Selected Essays, Phoenix 2003

Darwin, Charles, The Descent of Man, 1871, http://www.infidels.org/library/historical/charles_darwin/descent_of_man/

Darwin, Charles, The Origin of the Species, 1859, http://www.infidels.org/library/historical/charles_darwin/origin_of_species/

SOA in a Virtual World

noreply@blogger.com (Merkel Marmaduke) — Mon, 10 Mar 2008 07:23:00 +0000

Wikipedia defines a virtual world as a computer-based simulated environment intended for its users to inhabit it. Often a computer application is a business process made virtual by the use of computer technology. We therefore have to be careful when discussing the virtualization of computer environments as this may imply a virtual virtual reality.

The only example of virtual virtual reality I have seen is in a Futurama episode. It displayed an arcade booth with the label 'Virtual Reality' complete with a player using stereoscopic goggles and wired gloves. It then displayed another arcade booth with man sitting in a simple chair imaging he was wearing stereoscopic goggles and wired gloves. The caption to this booth was "Virtual Virtual Reality".

If we talk about "Virtual SOA" we must be talking about something that is not an SOA at all. Judith Hurwitz comes close to this when she discusses a "Virtualized SOA" in her article "Is virtualization the foundation of SOA?"

Andrew Doble writes about building a virtual SOA without investing in in-house software like an ESB. I would argue that this is in fact legitimate SOA. The only thing "virtual" about Andrew's suggestion is the ownership of the software tools.

Virtualization and SOA however do go together. Judith Hurwitz defines Virtualization as

"a technique for abstracting the physical characteristics of computing resources
in order to more easily leverage hardware systems, applications, operating
systems, networks, graphics, data or storage so they can be repurposed based on
customer need."

I believe that loose-coupling is the foundation of SOA but virtualization in many cases allows the benefits from this loose-coupling to be realised. The basic examples of virtualization are virtual servers, storage area networks and thin client platforms such as Citrix. These provide infrastructure flexibility for the deployment of SOA software. The design of the software for an application written using SOA is not affected. The SOA discipline should make sure that flexible deployment options can be used.

Manufacturers such as IBM are building business around SOA infrastructure. One thing to remember is that you do not need virtualization to build SOA. An SOA application can be built on a single server if you required, although it should also give you the flexibility to distribute this application and take advantage of virtualization options.

I agree with Lorraine Lawson in her posting "SOA and Virtualization: Complex, Fuzzy Ideas That Go Great Together? ". These two concepts mix well. However SOA is from the software development world and virtualisation is from the infrastructure world. We should beware of talking in terms of virtualizing our SOA. Our CIOs may consider we are sitting in a corner thinking we have an SOA when this is just a virtual reality.

Fry from "Futurama" with a virtual Lucy Liu.

The Importance of Meta-Projects

noreply@blogger.com (Merkel Marmaduke) — Sat, 23 Feb 2008 22:28:00 +0000

The feature video on International Association of Software Architects (IASA) site last week provided. I wish I could reference the speaker and the name of the video but alas last weeks feature video has been replaced by this weeks feature video.

The talk was reasonably pragmatic discussion of Architecture by a staff member of delta airlines. What inspired me to write this posting was that a slide that depicted "Business Enablements" things, like agility and collaboration. It also depicted lists for Business strategy and IT strategy. This is nothing new but it reminded me I need to lift my eyes from the day to day IT projects and see what my organisation is doing for software development as a whole.

This is timely because its time for my branch to put together a plan for next year. Typically this has been very project-focussed. Of course I also hope to put together a good graphic although I doubt whether I can do as well as the delta-airlines chart.

Each business and each IT department has its projects to achieve some outcome for the business. My focus is on application development projects. Often these projects are part of a bigger business project and commonly involve business process changes and infrastructure changes. Another category of projects do not achieve direct outcomes for the business but allow the business or the IT department to undertake these projects in a more effective manner. For business projects these 'meta-projects' is what was referred to as "business enablements" by the IASA speaker. I will not be using the term further as my spell checker says "enablements" is not a real word, and I tend to agree.

To put these in context see the following table

	Business	IT
Goals/Objectives	Business Goals. Mission, Vision and Strategy	IT Strategy
Meta-Projects	Business meta-projects	IT meta-projects
Projects	Business projects	IT projects

There are strong relationships between each cell in the table and its neighbouring cells. The business goals drive the IT strategy and also the business meta-projects. Business meta-projects may drive business projects but also meta-projects may be driven by projects. Neither the meta-project or project is necessarily the driver. This is the same with IT meta-projects and IT projects. IT strategy should drive the IT meta-projects and IT projects.

The IT Meta-Projects that my IT department is engaged in are:

SOA
Enterprise Architecture
Mobility (Supporting mobile computing devices)
Business Intelligence (Data Warehouse and Reporting)
Legacy Transition
Spatial Systems (Geographic Information Systems)
Online Project (Customer-facing IT systems)
Identity Management (Authentication/Privileges/Access Control)
Service Management (ITIL, Request Management, Change Management)
Performance Management (Individual Performance)
Cost Efficiency/Consolidation (ongoing)
Business Alignment (ongoing)

Some of these meta-projects have explicit planning documents explaining the plan. Others are more just acknowledged as the agreed direction of the IT department. Cost efficiency and business alignment are not specific projects but an ongoing focus of attention as they would be in most IT departments.

Some of these IT meta-projects such as Online Project, Service Management, Performance Management and Costs Efficiency have analogous business meta-projects. Business Alignment parallels the business meta-project of customer alignment.

This list of IT meta-projects has not previously been written out. It is surprisingly long, especially given the much longer list of specific software development projects that are on our books. Ideally each of these meta-projects should be explicitly planned, prioritised and monitored.

Meta-projects provide a foundation to the work an IT department does. They could also be referred to as 'foundation projects' but they are not necessarily just internal projects as they may be programs of projects that directly affect the business user. The meta-projects help build the look and the feel of the IT department. They establish its competitive advantage and its identity. Accordingly, it is important to get these meta-projects right and to provide them with the attention they deserve over the more day to day concerns of getting software out the door.

Architecture on Valentine's Day

noreply@blogger.com (Merkel Marmaduke) — Sat, 16 Feb 2008 11:20:00 +0000

I attended the Australian Computer Society (South Australian Branch) Architecture SIG this week. It happened that this was on Valentines Day. This drew a few comments about our dedication to the craft of IT Architecture, as we chose to spend the evening talking about architecture rather than with our chosen partners.

There were two presentations. Paul Turner introduced the International Association of Software Architects (IASA) to us. They have just set up chapter in Adelaide and will associate with our Architecture SIG.

IASA is an international professional society for Architects. It provides standards, training, certification and advocacy for Software Architects. Paul discussed some fundamental questions such as:

What is an architect?
What is software architecture?
How can yo prove you are a software architect?

Paul explained how IASA could help answer some of these questions. The main item of interest for me is that one can now be certified as an Architect. You pay for training over a number of months and then get grilled by a board which determines whether you have made the grade. This is an interesting development and will provide some legitimacy to the role and credentials of a software architect.

Adam Davies then gave a demonstration of Ruby on Rails. I had previously heard much about this technology but had not seen it in action. It was a very competent demonstration and it was amazing how much Adam was able to put together in the course the demonstration without resorting to the "here is some code I prepared earlier" fallback. Rails is able to set up sensible defaults to field names, column names and object names and create mappings between these without much overhead. It seemed to be ideal for standalone applications but certainly had some features that would make it valuable in any software development environment.

Coupling Calculus

noreply@blogger.com (Merkel Marmaduke) — Mon, 28 Jan 2008 07:46:00 +0000

I have had thread going through this blog on Coupling. Coupling is important because good definitions of SOA are dependent on the concept of 'loose coupling' or more precisely, 'message coupling'. The thread so far has included the following postings:

Cohesion, Coupling and SOA
Bragging Rights on Coupling
AOP: The Worst Form of Coupling
Binding and Coupling
Subroutine Call Coupling
Seven Levels of Coupling or Seven Deadly Sins?
Coupling with Contracts

In the first posting of this series I went back to the origins of the coupling term that was coined by Edward Yourdan to explain good programming practice in terms of subroutines. I then tried to come up with a list of coupling types which would help explain what were good and bad form of coupling.

This served to clarify the concept in my mind but the list of coupling types is not exhaustive and term 'coupling' is still not clearly defined. Coupling is a measure, usually described as 'loose' and 'tight' which correspond to the value judgement of 'bad' and 'good' respectively.

A good measure has numbers that describe it, but this is not yet the case with coupling. I and other authors have attempted to rank coupling types in order of merit with the ranking an implicit measure of coupling.

There have been attempts to measure coupling by looking at program artefacts like variable names and measuring how many times they are referenced. These attempts seem to miss the point by measuring what can be measured rather measuring coupling properly. Some of these approaches are described in footnote 1.

In this posting I try to suggest some definitions relating to coupling. This will not come up with useful numbers but it will try to explore the concept of coupling and may be a basis on which more theory can be developed.

Coupling is about how easy it is to change applications. If one part of an application (module) has to be changed because another part of a program changes then these program parts a tightly coupled. If one module does not have to be changed for changes made to another module then these application modules are loosely coupled. There are a number of programming practices that can be put in place to loosen the coupling between modules. This is good because it reduces the effort required to maintain your code.

The above would seem to be a fairly elementary description. Definitions however can still be made more precise, so that the concept can be analysed in more detail.

Lets say we have two modules A and B in a system S. The number of changes that can be made to A can be defined as C(A).

We can make changes to A because we want system S to do something different, or we can make changes to A for other reasons. We might want S to be produce results more quickly or we just might want A or S to be reorganised to be more easily comprehended. A change to A might not mean any change the output of S.

In order for A to be changed and the result of S to be changed to a desirable result (which may be an unchanged outcome for S) then in some cases some module B would have to change as well.

The coupling between A and B is defined as the effort required to make changes to B as a result of these changes A. For change j then there is work (and cost) associated with this change for systems S say Wj(S). The work in module Wj(A) and module be Wj(B).

Coupling then can be defined in terms of an individual change
Coupling(j) =the work required in B to make change j as a result of the change in Aj.
Coupling(A,B) is the sum of all work require in B for such changes j.

This leaves the concept of 'Coupling Type' to be explained. A system can be organised to reduce coupling between A and B. A coupling type is a way of organising the system. Some ways of organising will mean that a change to A require changes to B. Some ways of organising will mean that B can remain independent of changes in A. Some ways may increase or decrease the work involved in making changes to B when A is changed.

In general a way of organising a system S is O(S) which is another system which produces the same results. This is a change to the structure of the system rather than a change to the outcomes of the system. If we wanted to measure the coupling of a way of organising O then 'all' we have to do is sum all the Coupling(A,B) over all such S, A and B. A coupling type describes a number of ways of organising a system. The coupling associated with a coupling type is therefore the sum of all the Coupling (A,B) over all these ways of organising S.

To express this succinctly:

Coupling between two modules (A and B) of a system (S) is the work required to make changes to a module (B) as result of changes to another module (A) and still produce the desired outcome of a system (S).

A coupling type is scheme for organising the system (S). A measure of the coupling of a coupling type is the sum over all A and B for systems S which conform to this coupling type.

These technical definitions of coupling describe big, practically incomputable numbers. The conclusions we can draw though are

There is nothing really special about a coupling type. There could be a very large number of these and they are not confined to simple statements about how to pass parameters between modules. My earlier posting attempting to count these is inherently futile expect to provide common examples.
Coupling is about work relating to a change. I have not defined what I mean by 'work' intentionally. We could count lines of code changed or look at whether logic or data is changing. We count the costs of making these changes (eg. Hours of labour). It is meant to be used in its common sense. If it is necessary to trawl through technical documentation to make a change to one line of code this is still consider then the trawling effort should be consider as part of the work expended for this change.
Coupling is about systems in general. It is not just code instructions that are relevant. You might have to change constants, configurations, contracts or XML. These need to be considered in the concept of coupling.
The concept of coupling is relevant to systems theory examples outside of the field of computer science. Systems can also refer to biological or sociological systems (See footnote 2).
A change may not actually change the behaviour of the system at all but it is still relevant for coupling.
Coupling at its most elementary level looks at two particular modules, a particular system and a particular change. Thus it sensible to say A is tightly coupled to B in a system S in relation to change j.
Coupling can aggregated up to the module or level or to the Coupling Type level. Thus we can have A loosely coupled to B. We can talk about a Way of Organisation or a Coupling Type as facilitating loose coupling.
Restriction have not been placed on A and B. Therefore if B=S we can have A tightly coupled with the rest of the system S. A and B could intersect or A could be a sub-module of B.

It is still important to describe different coupling types and to determine what types promote loose coupling and what types do not. Good practice will reduce coupling for common programming changes.

To determine whether one coupling type is better than another the above suggests we need to enumerate all possible changes and work out which type leaves you with less work involved in one module as result of changes in another. This is obviously impractical but the analyst can do a couple of things to determine a good coupling type. A range of typical changes can be investigation or the coupling type can be looked at in relation to a specific type of change. For instance the benefits of message queuing can be looked at in relation to changing the location or speed of hardware.

This posting has added some precision to the concept of coupling. Counting all the possible ways to change a system is futile, however good and bad coupling will be become evident by looking at coding examples and foreseeing the changes that will be required to any system.

Footnote 1:

The following web pages describe metrics for coupling. These can be based on a static analysis of code or a dynamic tracing of the execution paths. In both cases the emphasis is counting the reference to a variable or type. This has some value in the task of maintaining code but is not a measure of the work required to change one module based on the need to change another. In some cases there may be some correspondence between the amount of work in changing the use of the variable with the number of times it is reference but not generally.

Lethbridge, Timothy C. and Anquetil, Nicolas, Experiments with Coupling and Cohesion Metrics in a Large System, April 2000, https://www.site.uottawa.ca/~tcl/papers/metrics/ExpWithCouplingCohesion.html

Poshyvanyk, Denys and Marcus, Andrian, The Conceptual Coupling Metrics for Object-Oriented Systems, http://www.cs.wayne.edu/~vip/publications/poshyvanyk-ConceptualCoupling.pdf

Glover, Andrew, In pursuit of code quality: Code quality for software architects: Use coupling metrics to support your system architecture, April 2006, http://www.ibm.com/developerworks/java/library/j-cq04256/
FrontEndArt describes one of its products for montoring source code quality include various metrics for coupling, http://www.frontendart.com/monitor/help/node22.html
Diederik Krols provides a blog-site archive which includes the postion "OO-Metrics for Coupling". This provides a coupling metric based on types shared across a module boundary. http://dotbay.blogspot.com/2006_01_01_archive.html

Footnote 2:

The concept of "structural coupling" in systems theory is used to describe co-evolution in biological and social systems. This is where, if entity A evolves then entity B might need to evolve to compensate. There would seem to be more complicated processes at play here than in a computer application.

Kay, Robert and Goldspink, Chris, Towards a Complex Non-linear Systems Theory of Organisation, http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-72/044%20Kay%20Complex.pdf

More on ESBs

noreply@blogger.com (Merkel Marmaduke) — Fri, 18 Jan 2008 01:52:00 +0000

One of my for more popular postings has been "ESB or not ESB". I suspect the only reason was the word "ESB" appeared twice in the title and the Google search engine gave it a high priority when people searched on this acronym.

This was cited in an Italian language blog posting by Andrea Gumina which also cited a good posting by Steve Vinoski . Steve's posting is perceptive and has many good comments on it.

Steve is commenting on an earlier posting by Patrick Logan. Patrick describes the concept of an ESB as a "mangy mutt" because it is a " grab-bag of mechanisms that form no coherent whole". I agree with this assessment but feel that the ESB concept will mature. The ESB may be a mechanism to provide some plumbing. Many organisations SOA efforts need that plumbing because the system delivery has a number of leaks.

My organization was galvanized into inaction and has not yet bought an ESB. However we are fairly typical I would guess because we want the vendors products to do 'everything' for us. IT is not our core business and we do not want to worry about location or services, security, integration, messaging etc. We want our developers to focus on business logic and data models specific to the business. If the vendor came up with an ESB that really did remove the need to spend time on software infrastructure (and did not charge a fortune for it), then we would be interested.

I would also like our developers to settle on a "one best way" to develop applications rather than the organization investing in training and support for a multitude of programming languages and technologies. Steve advocates using the right language for the job and not being confined to one language. This is a noble cause but it is hard work learning another language and becoming proficient in it. Most developers I know have the one language that they would prefer to write everything in, just as most people I know don't communicate in anything but English. If I weren't so lazy I'd teach myself Italian so I could find out what that Italian site was saying about my blog posting. Similarly I think many developers can't realistically be expected to be truly multilingual when it comes to programming languages.

The discussion of Steve's posting gets a hijacked by the REST vs Web Services debate. This is unfortunate because the ESB discussion should be a separate issue. Using SOAP and Web Services does not require an ESB. ESBs have been conceived to assist with REST such as open source ESBs WSO2 and Mule. The more popular arrangements seem to be that Web Services use an ESB and REST does not, but this need not be the case. The REST vs Web Services argument is taken up elsewhere and in some people's minds have already been settled. Ronald Schmelzer of Zapthink states:

In many ways, however, the debate about Web services and REST is as pointless as arguing whether a hammer or a screwdriver is a better tool.

In the discussion in Steve Vinoski's posting I found myself agreeing with Jonas Ekstrom and Paul H that the ESB market is not yet mature. Jonas put this eloquently:

Religions give easy answers, but there are no easy answers. Natural selection will strike again and again until we have reached the perfect SOA.

Paul H puts a similar slant on this:

I look forward to the day when we’re in a similar stage of maturity … and there exists generally accepted, standardized, manageable, and self-federating mediation technologies … until then, we’ll all have to suffer through endless arguments along the lines of 'should I use a BEA ESB or a Sonic ESB? is an ESB really necessary, or can I get away with REST + Dynamic Languages?'

I will continue to wait until the need for an ESB (or some part of one) becomes self-evident. While an implementation of SOA can function without and ESB there seems little point in making the investment.

Coupling with Contracts

noreply@blogger.com (Merkel Marmaduke) — Mon, 24 Dec 2007 05:10:00 +0000

I listened to a podcast by Thomas Erl yesterday which gave yet another slant to the subject of coupling in SOA. My previous posting about coupling was inspired by a Zapthink arcticle on "The Seven Levels of Loose Coupling" seven levels of coupling. The Thomas Erl podcast provides more information on at least three of the seven levels described by Ronald Schmelzer in the Zapthink article. These are

the coupling between the service consumer and the services implementation,
the coupling between the service consumer and the service contract, and
the coupling between the service consumer and the business process

In his podcast Erl asks the listener to refer to a diagram on his "SOA Principles" web-site. The types of coupling he focuses on in the podcast and the diagram are:

Service consumers are coupled to the service contract

The service contract can be coupled to a parent business process
The service contract can be coupled to the service logic
The service logic can be coupled to the service contract
The service logic may be coupled to proprietary vendor technology
The service logic may be coupled to multiple services it may need to compose
The service logic may be coupled to various resources that are part of the overall implementation environment

These coupling relationships broaden the possibilities explained in the Zapthink article. Firstly all the coupling relations are not based around the service consumer and secondly the coupling is not a symmetric relationship. Erl explains that it is good to couple the logic to the contract as this is contract-driven development but poor practice to have the contract tied to the logic of the service.

The multitude of coupling relationships was referred to in my earlier posting. When discussing coupling, it is important to be clear on what is being coupled when explaining service coupling. Zapthink discusses implementation, contract, policy, business process, data schema, infrastructure and semantic layer. What Erl describes is the service logic is part of what Zapthink describes as the implementation. Erl does not describe policy coupling. The data schema is part of the implementation environment and the infrastructure could be vendor technology.

I would prefer to keep coupling as a symmetric relationship. In other words if logic is coupled to contract then contract is coupled to logic. Erl is trying to cover methodological issues when he talks about contract-driven development. This is an important 'pattern' but it is not about coupling. It is about what a developer does first. The fact is that the contract and the logic are coupled and if this coupling can be made loose then that will provided more flexibility.

The focus of the Zapthink article is on the coupling with the service consumer. Erl has a twin focus on the contract and the service logic. My previous discussions on coupling tended to focus on coupling between services (or program modules). I tend to think that most important coupling concern in SOA is between the consumer and the service implementation in SOA. The consumer may be a user interface or another service. Contract coupling is important but it is more of a vehicle for loosening the coupling between consumer and provider.

One strength of the Zapthink article is that it has enabled me to analyse the Erl model. As is usual for Thomas Erl, his illustrations provide some clarity to a complex interaction of concepts. In this case his diagram should be taken as a sample of the coupling issues rather than a representation of all of the coupling issues relevant to SOA. Also as discussed above I suggest we stick with a symmetric notion of coupling to keep the world of SOA a simpler place.

Does normalization apply to Services?

noreply@blogger.com (Merkel Marmaduke) — Sat, 22 Dec 2007 11:01:00 +0000

I was listening to a podcast by Thomas Erl this week where he was talking about a pattern for normalization of services. This was interesting because there is not much in SOA with the same theoretical rigour as database normalization inspired by the work of Edgar Codd in the 1970s and the relational algebra and the Structured Query Language (SQL) that followed it.

The normalization that Erl talks about is fairly limited. On his website he summarizes normalization as

Problem
When delivering services as part of a service inventory, there is a constant risk that services will be created with overlapping functional boundaries, making it difficult to enable wide-spread reuse.
Solution
The service inventory needs to be designed with an emphasis on service boundary alignment.

There does not seem to be a lot of science involved in decomposing services to their basic elements so they can have maximum reuse and be easily composed into higher level services. Perhaps more precisely it is a 'soft' science that is all about governance, process and aligning with user requirements.

Normalization of services is not something that has been picked up strongly by other authors however David Chappell has his description of the normalization project which is consistent with Thomas Erl:

An idealized SOA environment provides a cleanly delineated set of services,
with one way to accomplish each function. This kind of normalization makes reuse
self-enforcing, since each service provides access to a specific group of
information in a specific way.

In his podcast Erl gives an example. If we have a service to get an invoice, then it is redundant to have a service or a method to get an invoice header. To have both is not normalized however there are reasons you may require both and therefore this would be 'denormalized'.

I am a little critical of the use of the term 'normalisation' to describe removing service redundancies. The processs of normalization in data modeling (and mathematics such as linear algebra) is a very precise and well developed technique where the concept used by Erl and Chappell while useful, is fairly loose.

If we wanted to do the equivalent of first normal form in service interfaces we might consider getting rid of the service equivalent of the "repeating groups" that destroys the relational model. If there are many 'invoice items' for an invoice, this means you could not accommodate 'invoice item' in your 'invoice' message. The strength of SOA relies on being able to exchange documents that mean something to the user. This is where attempts to get scientific about normalization fail the cause of SOA.

I will take this normalization discussion to its absurd ultimate. When I did my final year of my honours degree I simulated hardware that consisted entire of only two elements. They were the 'Nand gate' (logical not(and (a,b)) ) and a single binary digit of memory (Flip flop). The computer program that ran on this simulated hardware was simply a wiring diagram. Any computer program that can be represented in normal programming languages can be represented as a combination of these two elements. The sort of programs I ran on this simulation were the full adder, 4 bit timer and 3 bit shift register. Programming at this level is more a level in elementary digital electronics than it is in software development.

This was a distributed network of "services" at the very elementary level, but it is not very useful as a software development approach. This is a completely elementary normalized distributed application that can not be reduced further. This could be described as the ultimate implementation of reuse. You only have two 'services' (Nand and memory) both of which are reused constantly. The challenge is to link them together in meaningful ways. These services however are tightly coupled because of their synchronous nature and therefore this is not an SOA.

As a programming language this is worse than assembler language. Trying to wire gates together is even more detailed than assembler coding. Even if we could get SOA performing brilliantly at lower levels of granularity we do not want to go to such a low level of granularity that business entities are unrecognizable.

Beware dogmatic approaches to normalizing or simplifying a service inventory. I suspect the MEST architectural style with its focus on removing verbs or methods is heading down this type of reductionist thinking.

An overriding concern should be to meet business requirements and this might mean implementing redundant services and poorly structured business documents in services. Designing with a focus to eliminate redundancy is a good idea. It is nevertheless useful to seek out redundancies and investigate the option of reducing these if it does not compromise the ability of the services to represent business processes. Too much science in this process can reduce your services to absurd solutions.

Reference:
Kimber, Antony, The General Logic Array: An Abstract Architecture, Honours Report, The University of Adelaide, Department of Computer Science, November 1985.

This posting may have seemed a flimsy excuse to expose this work to the public domain but I hope it has described a useful anti-pattern for SOA.

The above project was done around the time Sun were having success introducing their Reduced Instruction Set Computer (RISC) chips which included simpler instruction sets that performed more quickly. Reducing down to a couple of elementary components is the extreme end of this strategy. There was also early interest in molecular electronics with the hope that if we could get molecules to behave as electronic components we could produce very small electrical circuits. One of the challenges of the project was working out lattice arrangements that would allow the appropriate connections to the two or more types of elementary components to be made.

Book Review: Loosely Coupled - The Missing Pieces…

noreply@blogger.com (Merkel Marmaduke) — Sun, 16 Dec 2007 04:07:00 +0000

This is a review of the book "Loosely Coupled: The Missing Piece of Web Services", by Doug Kaye. In a previous posting I expressed delight that an author had dedicated a book to this important topic. This perhaps is not the definitive book on loose coupling but there is plenty of value for the reader.

Doug has an easily readable style and sticks to a high level but there is enough technical substance to keep and IT architect or IT manager interested. You will find no code snippets and no programming standard specifications described in this book but there are some good diagrams and deep insights into SOA and how an organization should deliver SOA.

This book was published in 2003 and I kept expecting the information to be dated. There was no mention of an Enterprise Service Bus (ESB) but Doug Kaye describes a number network models, mediation tools and deployment options which have been incorporated into what vendors now label as an ESB. More importantly some of the missing pieces he describes in 2003 are still missing at the end of 2007. Security, transaction integrity and business semantics still do not have accepted solutions within SOA and these missing pieces are described in this book. The WS-* standards have progressed somewhat since the writing of "Loosely Coupled" but basics of SOAP, WSDL and UDDI were in place and many for the WS standards are foretold.

Doug has good perspective on what is required in the corporate world. He advises everyone to have your "elevator pitch" ready for that occasion when the CEO gets into the elevator with you and asks something like "So what's this I keep hearing about web services? What are they and why are they so important" (p27). Doug offers a response that suggests he is either a fast talker or works in a building that is much taller than mine, but the concept of having something ready like this is important if you are going evangelize about web services.

There is only one chapter dedicated to Loose Coupling and Doug has not tried to say everything there is to say on the subject. This justified in his statement "…loose coupling is a methodology or style, rather than a set of established rules and specifications" (p. 131). Figure 10-1 provides a table describing 10 types of coupling and descriptions of tightly and loosely coupled approaches to each. This is a good approach as it is not easy to describe coupling succinctly. I found this approach in another posting although interestingly the contents are different. I have found that in my previous posting that it is difficult to label types and coupling and the three column approach will enable me to do this more effectively. The book covers late binding, heterogeneity, routing and transformations well.

It is true to say that this book is more to do about its subtitle "The Missing Pieces of Web Services" than Loose Coupling. Doug makes the bold prediction that when we have solved all the pieces of the puzzle in the future that it will be so commonplace that "Web services won't be mentioned by the IT trade press, and the topic won't be tracked by analysts". The book searches through these missing pieces in most of the chapters. The fact that Web Services is still a strong topic for analysts and vendors is evidence suggesting that we have not yet found all those missing pieces.

Doug Kaye provides a good description of transaction control although I found the book I read earlier by Krafzig et al to have more detail in this area. Doug has two strong chapters on the problems of web services security and covers this topic in a thorough and systematic way. A couple of reviews have suggested that the book by Mark O'Neil also provides good coverage of Web Service security.

The chapter on deployment option talks about commercial opportunities for third parties in the delivery of services and services infrastructure. This discussed Web Service Networks, Distributed Web Service Networks, Web-Service Delivery Networks and Web Service Providers. Doug takes the service vendor view in his final chapter on providing external services. Both these chapters had little relevance to me. Four years after this book was written I have not been flooded with vendors wanting to sell me services or wanting to host infrastructure for me to operate my web service applications. Perhaps the money is not there, or perhaps it is just easier to use open source products or set up our own infrastructure. I still think Web Services provides some opportunities for commercial intermediaries, but until the vendors are out there and dealing in the government sector it will be difficult for me to get excited about this topic.

Doug writes a couple of good chapters on the management of simple and complex web services projects. In particular he deals with timing the project around web services approaches that have not yet been delivered. The chapter on Service Level Agreements (SLAs) was an important piece of the completion of the web services picture.

The book concludes with a number of checklists that anyone embarking on an SOA project should refer to. This should help the SOA project manager get their ducks in line to navigate the difficult road ahead.

In summary, this was a well written book about the evolving phenomenon of Web Services. There are some valuable insights into Loose Coupling but an exhaustive technical treatment of coupling and the programming patterns that can reduce coupling is out of the scope of this book. There are some important insights to be found, especially on the subjects of security, evangelizing and commercial arrangements. I expect I will refer to this book many times in the future.

Loosely Coupled: The Missing Pieces of Web Services, Doug Kaye ; RDS Press 2003, ISBN 1881378241

Seven Levels of Coupling: Seven Deadly Sins or Seventh Heaven?

noreply@blogger.com (Merkel Marmaduke) — Sun, 02 Dec 2007 00:46:00 +0000

In this posting I discuss the Zapthink article by Ronald Schmelzer, "The Seven Levels of Loose Coupling".

This week I have found two commentators redefining loose coupling. The first is described in the discussion about virtual endpoints that was the subject of my previous posting and the second was this Zapthink article. I think it is important we clarify our definitions around SOA to avoid the concept being hijacked by commercial interests. So in this posting and the last I have attempted to sort out whether the discussion is adding to our understanding or muddying the waters.

In a previous posting I have identified 18 different types of coupling and I am still finding more. The first thing to realise about the seven levels of coupling is they are looking at a different attribute of coupling than what I refer to as 'types of coupling'. The levels refer to the entities that are being coupled. My discussion on coupling types looks at the coupling of two modules of program code. This code could be in calling programs, objects, subroutines or services.

The following table lists the seven levels. This table makes the distinction between 'Levels of Coupling' and 'Types of Coupling' clear. The core tenet that Zapthink is discussing is the "loose coupling of Service consumers and providers". This has a different meaning than loose coupling between program code in different services. The main difference is that aspects of the service provider, with the exception of the implementation are not program code at all. Another difference is that a service consumer may not be a service either and it also might not contain program code.

	Aspects of the Service Provider	Possible Types of Coupling
Loose Coupling the Implementation	The service implementation	18 types and still counting
Loose Coupling the Service Contract	Service contract	Interface Coupling, Bind-Time Coupling, Service Intermediary Coupling, Infrastructure configuration Coupling
Loose Coupling the Service Policy	Service policy	Bind-Time Coupling, Service Intermediary Coupling, Infrastructure configuration Coupling
Loose Coupling the Process	Service-oriented process (Trivial if this is a service)	18 types and still counting
Loose Coupling the Data Schema	Data schema of provider dataset and data schema of interface.	Data Services Layer Coupling, Data Transformation Service Coupling
Loose Coupling the Infrastructure	Infrastructure of provider	Single-vendor SOA platform Coupling
Loose Coupling at the Semantic Layer	Meaning of message attributes	Dynamic service definition Coupling

The seven layers are not as clearly layered as the phrase 'Levels' might suggest. Certainly policies could be considered a higher level than contracts and processes considered at higher level to implementation. The other levels could be considered to sit atop the infrastructure level and under the semantic level but normally data and process would sit beside each other with neither the master. I also see contracts sitting beside data and process as equal partners.

Coupling takes account of the dependency between two entities and the clarity of this dependency. Thus two services exchanging a message are loosely coupled if every dependency is clear from the format of the message. Services have different reasons for being dependent. These reasons are what we mean by 'Types of coupling'.

The types of coupling for the seven levels are described in the third column of table above. The first level describes service to service coupling which I have described earlier. The fourth level is interesting because the solution is almost trivial. The business process should look like a service and therefore all the coupling types for coupling between services also apply for coupling between a consumer service and provider service that is a composition of services used to implement a process. The sort of things that might cause dependencies with contacts, polices, infrastructure, data schemas and semantics are different to the program code dependencies.

Loose coupling of data schema is partly addressed by another Zapthink article which advocates the building of data transformation services rather than incorporating transformation logic inside services used to contain business logic.

Tight coupling between the consumer service and the infrastructure of the provider could be based around a number of proprietary technologies. The ones highlighted by Zapthink are the non-standard ESB and the single vendor SOA platform.

The types listed in the third column of the above table are those implied in the Zapthink article. I expect that a thorough investigation would uncover other types of coupling as well. The point of listing them in the table is to suggest that each coupling level will have its own types and each level (with the possible exception of the process level) needs to be investigated separately.

We can find coupling examples everywhere. For example Carlos Perez and David Walend discuss whether a DBMS and its application are loosely coupled if the application is accessing the DBMS using SQL. This is interesting but not relevant to SOA. For the point of view of defining SOA, I do not believe the seven levels of loose coupling matter. You can have an SOA as long as the services are loosely coupled. However as a model to assist good practice, I think the seven levels will provide value even if they do overload the 'loose coupling' term.

The concepts discussed in relation to levels of coupling are certainly valid. We do not want service consumers dying or behaving incorrectly due to changes with service provider's infrastructure, policies or any of the other aspects described in the seven levels. Zapthink continue to guide us on the path to software heaven, however in the process of introducing 'levels of coupling' Zapthink commit the sin of causing confusion around the important 'loose coupling' term. I recommend the "Seven Levels" article to SOA developers and I suggest developers need to be clear on what is being coupled whenever they use or encounter the term 'loose coupling'.

End Note:
One way to distinguish other levels of coupling from service to service implementation coupling would be to call the different levels something different other than 'loose coupling'. The names that occurred to me were 'loose buckling' or even 'loose zipping' which would be reminiscent of the zapthink integration zipper. Unfortunately both these terms have implications of embarrassing failure if the buckling or zipping is too loose. We do not want our metaphorical trousers around our ankles.

Virtual Endpoints: A Loose Definition for Loose Coupling?

noreply@blogger.com (Merkel Marmaduke) — Sat, 01 Dec 2007 02:28:00 +0000

I listened to a podcast this week by Miko Matsumura, Deputy CTO of Software AG. His comments on Governance were interesting especially the concept of hot and cold parts of the architecture. Some things like policies should remain "cool" and not change often. Other parts such as how you combine your services into applications should be relatively "warm" and change more often.

The point that jumped out at me was his very concise descript of loose coupling "Loose Coupling has virtualized endpoints". No further definition was given.

This must be a bit of a Software AG thing. John Michelsen, also of Software AG says

…virtual endpoints are a great idea as you think about the scalability and the loose coupling we want in a SOA.

This is better explained in John's posting "An Intro to SOA and Virtualization" but all this virtualisation seems to be is getting rid of hard coded URLs. This does not seem terribly revolutionary and is certainly not synonymous with the concept of loose coupling that is, according to some writers, the whole reason behind the benefits of SOA.

Before I discount this as an overly narrow vendor view I would like to explore the concept a bit more deeply. If I look down my ever-increasing list coupling types I can see a number of properties of service interfaces. I define and endpoint as being a service interface then we can see an endpoint has having the following properties and we can see some corresponding coupling types:

Endpoint Property	Coupling Type (From previous posting)
URL	Registry Coupling
Platform	Platform Coupling
Interface Time	Queue Coupling
Processor Location	Distributed Coupling
Data Field Order & Presence	XML Message Payload Coupling

The only property that may need some explanation is "Interface Time". This is the time that a message leaves from or arrives at a service interface. If no queuing is involved then the only delay between the message leaving from a sending service and arriving at the receiving service is some network latency. If a queue is involved then the delay uncouples the sending time from the receiving time.

The concept of an endpoint does not have much meaning except where messaging is involved. There are some types of coupling in the messaging world that are not covered by the concept of virtualised endpoints. Coupling based on process state (including synchronous coupling) and coupling introduced to manage process control, such as transaction control is not easily expressed in terms of endpoint properties. Logical coupling where service behaviour changes significantly based Boolean or enumerated field values passed in the message also is not covered as an endpoint property.

The "Virtual Endpoint" concept turns out to be a fairly useful. We have some types of coupling that are only dependant on endpoints (or service interfaces) and others that depend more deeply on the implementation of the services or the interpretation of the messages being passed. Virtual endpoints are not the whole story when it comes to loose coupling but are part of it.

Subroutine Call Coupling

noreply@blogger.com (Merkel Marmaduke) — Thu, 22 Nov 2007 12:30:00 +0000

It has been more than a month since I did my last posting on coupling. Amazon has delivered my book on loose coupling by Doug Kaye and I am finding it valuable. Once I have finished that I hope to revise my original list of types of coupling.

In this posting I would like to say something about subroutine call semantics versus message passing and the coupling associated with this. I have discussed before that the synchronous approach which is a requirement of the subroutine call semantics is more tightly coupled than an asynchronous approach. There are two other differences between subroutine calling and message passing. These are

the management of state of the calling module and the called module; and
the management of the called instance itself.

When you invoke a subroutine it starts executing from its beginning. The invoked routine will usually have no state and complete before the invoking routine resumes again. Another form of routine call is the coroutine where the state in the called module is maintained from one call to another. The issue of state coupling was covered in my earlier posting and labelled as 'Process State Coupling'. This type coupling is intended to cover coroutine behaviour. A coroutine is even more tightly coupled than a normal subroutine call because of the dependency on state.

In a normal subroutine call state is not preserved in the called module from one call to the next however state is preserved in the calling module. It preserves its state by the simple expedient of suspending execution until the called module returns control.

Asynchronous message can still involve process state coupling. More loosely coupled messaging has enough information in the message to process without retaining state. This is, in general, better practice but is not always practical.

There is one other aspect of subroutine call behaviour that is different from message passing behaviour. When a subroutine is invoked it is in effect brought to life at that point. If you have to create an instance of a service in order to use it then this is another form of dependency. A service should be an entity that always "exists" that is ready to serve. In effect messages can wake up receiving services but it should not be the concern of the sending service to create or dispose of an instance of the service. I therefore intend to add 'Creation Coupling' to the coupling list when I next publish one. Creation coupling occurs in Java classes when a new object is created using the 'New' statement. Methods of that object are then called but the creation of new object (and sometimes its destruction) is often the role of the calling object.

It is the system's responsibility to effect and manage the invocation of the service, not that of the calling application. This allows two critical characteristics to be realized: 1) The services are truly independent, and 2) they can be managed. (Anagol-Subbarao)

In the paper "Web Servces Are Not Distributed Objects" by Werner Vogels he draws this distinction clearly. Loosely coupled web services do not have the same behaviour as objects.

An important notion at the core of distributed object technology is the object life cycle:
· Upon request, a factory instantiates the objects.
· The consumer who requested the instantiation performs various operations on the object instance.
· Sometime later, either the consumer releases the instance or the runtime system garbage-collects it.

It is this point that Werner distinguishes between services and objects and is also this point that makes distributed objects more tightly coupled than web services that are coupled using messages.

In this posting we have discussed subroutine semantics and discovered that subroutine a tightly coupled to their calling modules because of:

Synchronous Coupling
Process State Couping, and
Creation Coupling

The use of asynchronous messages is much more loosely coupled as a result.

References:
Anagol-Subbarao, Anjali, J2EE Web Services on BEA WebLogic, Prentice Hall, October, 2004

Vogels, Werner, "Web Services are not Distributed Objects: Common Misconceptions about Service Oriented Architectures" , IEEE Internet Computing, vol. 7, no. 6, pp. 59–66, 2003

This is also available from Werner's blog site at
http://www.allthingsdistributed.com/historical/archives/000343.html

Marlin, Christopher D. Coroutines, Lecture notes in computer science, vol. 95, Springer-Verlag, Berlin, Heidelberg, 1980. ISBN 3 540 10256 6 and ISBN 0 387 10256 6.

Kaye, Doug, Loosely Coupled: The Missing Pieces of Web Services; RDS Press 2003, ISBN 1881378241

Code Abstractions and Babushka Dolls

noreply@blogger.com (Merkel Marmaduke) — Wed, 21 Nov 2007 12:10:00 +0000

There are various ways of encapsulating program code. We encapsulate code so that it can be invoked from some other program code. The encapsulated code can be used in its abbreviated form. I will also refer to it as an abstraction in this posting.

There are some ways of encapsulating code that encapsulate a small amount of code and are fine grained. There are some ways of encapsulating a large amount of code or other abstractions of code and are coarse grained. I would like to be able to list all the forms of encapsulation from smallest to largest and arrange them like Russian Babushka dolls with a clear scheme for which goes inside which.

At the finest level of code abstraction I will start with functions, procedures and methods. They are pretty simple. The name 'function' has a mathematical significance the pre-dates computers. Procedures, subroutines and methods are all variations on the theme of a group of instructions.

The next level of abstraction is the 'class' or the instance of that class the 'object'. Object is an unimaginative name. We may as well have used the synonymous term 'thing' but that would have sounded like we just did not know what to call this abstraction of code. "Let's build a thing to access our database" sounds a lot less specific than "Let's build an object to access our database".

Then we have services. These should be reasonably coarse grained. They may contain classes which may contain methods. I skip java beans in this abstraction discussion that would seem to inhabit the same level in the hierarchy as services.

My metaphor of the Babushka dolls is already looking a bit shaky. There are methods that can contain methods, classes that contain classes. Recursion does not work very well with Babushka dolls. Also there are abstractions like code libraries and include files that cut-across the above abstractions, but I will press on anyway.

The level of abstraction I would like to look at is the component as described by the Service Component Architecture (SCA). Component is another synonym for 'thing'. Instead of 'object' and 'component' we may as well have used 'thing 1' and 'thing 2' and apologized to Dr Seuss for using his naming convention. According to the SCA we should then consider encapsulating our components in 'composites' and our 'composites' in 'domains'.

'composites' and 'domains' at least are getting way from being synonyms of 'things'. A composite is just a group of things and a domain is somewhere you find things in, a bit like a container. Of course a 'container' is typically seen as more a runtime environment than a way of collecting together code or abstractions.

'Domain' is a rather overused term in IT at present. Do not try to draw any parallel between an SCA domain, a Domain Name Service (DNS) or Domain Driven Design. There is very little connection.

I would like to bring the concept of Roger Sessions' 'Software Fortress' in at this point. At least Roger has come up with an imaginative name for his abstraction. His explanation of the various roles that software plays in relation to the Fortress is also imaginative as these are illustrated by various goblins, fairies and ogres in his documentation.

Definition: Software Fortress
A software fortress is a conglomerate of software systems consisting of potentially many processes running on potentially many machines spread out over a potentially wide geographic area that:
I. Serve a common purpose
II. Are owned by a cohesive group of individuals
III. Work together in a trust relationship.
IV. Are built on a common technology base.

Roger could improve his definition by indicating whether "or" or "and" separated the different lines of his definition. Definition line IV suggests the Fortress is quite similar to the Domain concept from SCA. If we include the other three definition lines we may find that a Fortress has finer granularity than the domain and may be closer to a composite or component.

The recommendation provided in a couple of presentations I have seen from Glenn Smyth is that web services should be built if and only if they are required on the border of a software fortress. The essence for the SCA is that a service appears on the border of all components. Therefore the finer grained interpretation of 'Software Fortress' that incorporates an 'and' between the last lines of the definition would seem more consistent with this view.

One difference between ways of encapsulating program logic is the stage in the software development life cycle. At runtime then objects, services and containers encapsulate running logic. At development time classes, include files and functions are relevant. At requirements definition time the domains in domain driven design have relevance and the "common purpose" aspect of a fortress helps define the encapsulation. We therefore would more correctly have three sets of Babushka dolls.

SCA is all about runtime and development time. There is nothing that suggests what part of the design a component represents. Domain driven design is all about business requirements. There is nothing in this process that advises how you are to encapsulate your code to represent the domain. SOA attempts to span all three stages of the development life cycle both by suggesting business services and then keeping to this granularity in the developed code and the runtime.

The encapsulation and abstraction of logic is a complicated area. I probably have not made it any simpler with this posting. It is increasingly vital in software. There is not much excitement as there used to be around programming languages but how we bundle together program instructions is still an area of active interest. The point to take away from this is that we have to define our terms clearly and use them in the context of the stage of the software development lifecycle. A consistent approach from design through to runtime would be nice, but at least your software development methodology should identify the mapping between different ways of abstracting business logic thus linking your three or more sets of Babushka dolls.

References

David Chappell, Introducing SCA,

http://www.davidchappell.com/articles/Introducing_SCA.pdf

Eric Evans, Domain Driven Design,(WebCast)
http://www.infoq.com/presentations/model-to-work-evans

Eric Evans, Domain Drive/Model Driven
http://domaindrivendesign.org/discussion/blog/evans_eric_ddd_and_mdd.html

Roger Sessions has book on Software Fortresses which I confess I have not read yet but the following article incorporates and overview of this concept and the newsletter below explains some of its origins.
Roger Sessions, Modeling Software Architectures
http://www.objectwatch.com/PaperAndSpreadsheetVersion1-0.zip

Roger Sessions, The Software Fortress Model; A Next Generation Model for Describing Enterprise Software Architectures, ObjectWatch Newsletter # 36, November, 2001

http://www.objectwatch.com/newsletters/issue_36.htm

Glen Smyth's presentation to the ACS is now available at
http://sa.acs.org.au/it_arch/index.php/Presentations_Past%2C_Present_and_Future