Blogger: Richard Watson

How do I know my BPM program is improving my business?  How do we know if conditions are improving in Afghanistan? In each case, there is only one way: You measure the right things.

If you are a regular reader, you'll know I'm not only interested in business processes, but I also have an interest in foreign affairs, especially in the conflict in Afghanistan. In a series of articles [1] for Foreign Policy, Thomas Ricks discusses a paper by counter-insurgency expert David Kilcullen which looks at metrics in counter-insurgency campaigns. When Ricks quoted Kilcullen discussing non-obvious metrics, such as "Transportation prices" and "Prices of exotic vegetables" this made the connection to the BPM field research findings I am documenting [2]:

"… to understand this weird [metric], you need to understand local conditions. What people are paying for vegetables grown outside their district is a quick indicator of road security. Trucking companies factor in the risks they face, as well as the cost of bribes and other forms of corruption. So variations over time may be a helpful indicator of trends in public perception of security conditions and the corruption level of government security forces."

The example of imported vegetable prices is a metric that may not be obvious but is having a real effect on the local people, and is connected to the outcome required from the counter-insurgency. Safe passage to and from markets is an indicator of commerce returning which heralds a return to normality for local people. Taliban insurgency will be sidelined by normality.

ISAF photo by U.S. Navy Petty Officer 2nd Class Aramis X. Ramirez.

What not to Measure

The key thing about those counterinsurgency metrics is that they facilitate tracing a line from the desired outcomes to measurements. The greatest challenge for BPM is that organizations have difficulty clearly articulating the value they derive from it. One key finding of the study is that participants find it hard to recognize, measure, and sell successes, indicating a lack of metrics aligned with business outcomes. One participant told us: 

"Management is not asking for any BPM metrics - for them – 'gut rules'."

The table below highlights the difference between metrics that are connected to business outcomes and those that are unconnected.

Increasing the number of automated processes is a success metric pursued by one of the BPM teams we spoke to. It is a good metric by most "SMART" standards. We are always told to aim for these criteria: it is (s)pecific: an increase of 1000; it is (m)easureable; it is (a)mbitious but (a)chievable: if the IT department is given the budget and sponsorship to do it; it is (t)ime-bound: by the end of 2011, but it is not (r)elevant or (r)esults-oriented because it is not connected to a business outcome.

Similarly, Ricks discusses bad metrics, such as enemy fatalities. 'Body count' is a SMART, but short sighted metric. As Ricks, reporting Kilcullen, says, "when you have 100 enemies and kill 20 of them, you may wind up with 120 live enemies, because you just created 40 more. It's more algebra than arithmetic."

One BPM field research study interviewee started their BPM effort by asking their customers what they need to complete a process and how they define "good service". Starting process analysis with the business outcome desired is a rare, but welcome approach. The majority of BPM teams we spoke to had strategic aspirations for their initiatives, but could only quote tactical success metrics.

Breaking Through to Strategic Wins

The reason Anne Thomas Manes published the blog posting "SOA is dead; long live services" in January 2009 was to highlight that SOA initiatives Burton Group studied were failing to articulate and deliver business value beyond "doing SOA". Many people think the implicit benefit of BPM is that you can "do BPM". The inability of a majority of participants in the study to articulate business value they derive from their BPM efforts is a concern for the viability of those efforts. Metrics should very clearly articulate the value of BPM. The counter-insurgency metrics are not just interesting indicators, as Economist blogger Free exchange says:

"these metrics strike me as more likely than to encourage the right strategic thinking. Asking yourself the question, "How do we reduce transportation prices," is more likely to produce strategy leading to a better Afghan outcome than asking, "How do we minimise coalition deaths?" And it may well actually minimise coalition deaths."

Encouraging that kind of strategic thinking is the source of richest value of BPM. In the field research study, the claim that one interviewee's BPM program "is adding value by breaking through assumptions and seeing what people do to add value" is the closest any of the participants came to sharing a lasting strategic result of their efforts, in this case a changed organizational mindset. 

This mirrors the current thinking of some foreign policy analysts. Writing in the Financial Times today about the need to focus on a better outcome from the latest "surge" in Afghanistan, Philip Stephens [3] says, 

"Operation Moshtarak carries the risk that tactical gains will be mistaken for strategic success. This is not the time to go deeper into Afghanistan. Rather it is the moment to think harder about withdrawal."

Our customer-focused BPM interviewee expressed this business optimization sentiment another way, 

"Sometimes we need to sub-optimize individual processes to improve the business overall."

---

[1] Thomas E. Ricks. Foreign Policy. Published 8-9 Feb 2010. Accessed online 16 Feb 2010.

http://ricks.foreignpolicy.com/posts/2010/02/08/kilcullen_i_here_s_what_not_to_measure_in_a_coin_campaign

http://ricks.foreignpolicy.com/posts/2010/02/09/kilcullen_ii_how_to_tell_the_effect_of_your_operations_on_the_population

http://ricks.foreignpolicy.com/posts/2010/02/09/kilcullen_iii_how_to_take_the_measure_of_an_afghan_official

[2] The findings of the BPM Field research study will be presented in Prague, April 19-22 at Burton Group's Catalyst Conference.  The first 3 parts of the 6 part series of field research findings will also be published (for Burton Group clients) in April:

• Field Research Summary: Business Process Management
• Field Research Results: Becoming Process Oriented
• Field Research Results: Architectural Foundations for BPM

[3] Philip Stephens. "Britain needs an Afghan exit strategy". Financial Times. Published 16 Feb, 2010. 

http://www.ft.com/cms/s/0/dc5e92f8-1a66-11df-a2e3-00144feab49a.html

As Anne Thomas Manes and many others commented on yesterday, IBM announced their intention to acquire BPM infrastructure vendor Lombardi.

An Opportunity for IBM

IBM was missing the business empowerment and collaboration pieces from their BPM story. Now they’ve got them. No question about Lombardi’s leadership on those fronts.  The question is how to weave these capabilities into a coherent, believable platform for influencing outcomes using processes, not just "a grab bag" (as one of our clients calls typical stacker BPMS products). And what an embarrassment of riches IBM has now:

Recession-proof?

The BPM infrastructure market was looking recession-proof. On analyst briefing calls this year, everyone's revenue numbers were heading the right direction. The leading pure-play vendors were all talking about operating from cash. So, was the business model shakier than we thought? Has the credit-crunch felled yet another solid-but-credit-strapped company? Did the VCs panic and hit eject? We don't know, but Miko Matsumura's opinion is that "undisclosed sum" is code for forced sale. If, and this is a big if, Lombardi was compelled to sell itself, where does that leave the other pure-plays, even the leaders like Pegasystems, Appian, and Savvion?

The problem for IBM's salesforce: go deep or wide?

One of the differentiators of pure-play anything is their single-mindedness. An intrinsic part of a pure-play BPM sale is rolling out the pure-play CXOs to meet the prospect's business managers. Lombardi's prospects get to meet Phil Gilbert and Rod Faveron up close and learn from their "we-live-and-breath-BPM-experiences". With all due respect to Phil and Rod, and of course we don't know their post-handcuffs plans, that's not the way IBM are likely to do a sale. Of course there is an executive responsible for BPM within IBM, right now he is Craig Hayman. Craig is a senior guy, but he's not Sam Palmisano.  The point is that the IBM salesforce will have more difficulty selling their passion for BPM outside their traditional IT audience.

"Take two bottles into the shower? Now I just wash and go."

Product coherence is a real issue for BPM buyers. Many organizations doing BPM have matured to demand infrastructure that handles different process types seamlessly. Using the "process, information, people" triumvirate, to describe how to approach BPM based on 3 different products won't wash. The teams we have spoken to have said that an offering of WebSphere Process Server plus FileNet just isn't an attractive proposition from portfolio complexity point of view, regardless how well each product handles specific process patterns. Take another discrete BPM product family into the mix? To make this work and become a leader, IBM has a serious product integration job to do. I don't doubt it can do it, if it has the will. The short-term, more-revenue-for-three-products strategy is a short sighted one. Like SOA, BPM is a long game.  You need to bring your customers with you, not make them feel more aggravated about maintenance fees over time.

Portfolio Overlaps

Sure there are some overlaps: IBM's BlueWorks BPM which is built on Lotus and Lombardi's BluePrint are two SaaS collaborative modeling environments. Something has got to give here. The remaining product or combination of these will be fighting a battle to be the go-to modeling environment with whatever results from Software AG and IDS Scheer's mashup of ARIS and AlignSpace. Our research shows clearly that the value of BPM tooling  is heavily biased toward the modeling environment –using the process model as a nexus of cross-functional business improvement discussions.  So this is the key battleground to watch in the next 12-18 months. Another overlap is recently updated WebSphere Business Monitor and TeamWorks' BAM capabilities which are both pretty sexy. Something's also got to give here. Planning another acquisition like Lombardi, may explain partly why IBM has not developed the Holosofx tools into a fully-fledged business-focused modeling environment over the last couple of years.

Questions in customers minds

Acquisition always makes the acquired's customers nervous. Things change – just ask BEA customers about maintenance fees and Eclipse tooling support. One price of entry for pure-plays is engineering their engines to run in a wide range of runtime containers. If my enterprise standard application server is WebLogic, I need your BPM product to run on WebLogic. I'm not going to certify and engineer another application server in my environment for you. How long will runtime aspects of TeamWorks be deployable on a range of application servers? What kinds of "optimizations" are planned for running the TeamWorks process execution engine on WebSphere AS? Vendors making a platform acquisition always counter with "well it would be crazy to cut off a revenue stream, wouldn't it?" And yes it would, looking a BPM adoption strategy lens. Viewed through a "WebSphere as the only platform you'll ever need"strategy lens, support for WebLogic and JBoss looks less of a no-brainer.

Blogger: Richard Watson

It was six men of Indostan 

To learning much inclined, 

Who went to see the Elephant 

(Though all of them were blind), 

That each by observation 

Might satisfy his mind 

The First approached the Elephant, 

And happening to fall 

Against his broad and sturdy side, 

At once began to bawl: 

"God bless me! but the Elephant 

Is very like a wall!"

The Second, feeling of the tusk, 

Cried, "Ho! what have we here 

So very round and smooth and sharp? 

To me 'tis mighty clear 

This wonder of an Elephant 

Is very like a spear!"

And, so on; the third blind man believes the elephant is a snake, the fourth a tree, the fifth a fan, the sixth a rope. I'm sure you get the picture.

And so these men of Indostan
Disputed loud and long,
Each in his own opinion
Exceeding stiff and strong,
Though each was partly in the right,
And all were in the wrong!

From "The Blind Men and the Elephant" by John Godfrey Saxe (1816-1887)

Over the last few months I have done a lot of talking to clients about their business process management (BPM) efforts.  Carrying out Burton Group’s field research study, I have spoken to 35 people representing 23 organizations.  I have probably gathered as many definitions of BPM.  BPM is the most extreme case of the blind men and the elephant fable I have experienced in my career.  How people think about BPM depends entirely on their point of view, their skills, and their experiences.

The situation is not as simple to characterize as "IT people believe BPM is about tools for automating processes and business people believe BPM is a management discipline".  The people we spoke to leading BPM initiatives have more complex and subtly differing viewpoints that are shaped by their experiences.

It's partly the rich heritage of BPM that leads to this variety of viewpoints across our enterprises.  When we examine the management, scientific and technical trends that have helped shape where we are with BPM today, and the different roles we play, it's not so surprising we all have differing perspectives. I've represented many of those trends in the diagram below, with the influential work[1] for each trend mentioned in the green clouds.

Burton Group's definition of BPM is:

BPM is a discipline for managing business processes explicitly as strategic assets.

The commitment to manage processes explicitly, and treat processes as strategic corporate assets, should be taken independently from any decision to automate part of the workflow.  So, BPM infrastructure is not a prerequisite for doing BPM. Process improvement thinking should come before adopting tools (more on this topic soon.)

Burton Group's approach to providing research and advice for our clients is always to focus on practices rather than technology.  Analyzing BPM demands this approach, because the tools play very much a supporting role to the organizational change required for optimizing the business through a process lens.

Why haven't the differing viewpoints on BPM converged?  Other questions remain: If process improvement has climbed to the top of CIOs agendas, why is business process management (BPM) still executed in enclaves? Even amongst organizations moving forward with BPM, why is there universal difficulty in articulating its business benefit?

Next week the Burton Group BPM team meets face-to-face to harvest the insights that have come from those interviews with BPM leaders.  During the consolidation session over several days, we will analyze the 1000s of data points and group them into patterns and relationships.  We will find answers to the questions above and more besides.  I'm looking forward to publishing the findings early next year, and sharing them at our Catalyst User conferences: Prague in April, and San Diego in July.

[1] Two references were helpful in compiling this timeline: "Business Process Change" by Paul Harmon. Morgan Kaufman. 2nd Edition. 2007 and "Workflow-based Process Controlling" by Michael zur Muehlen. Logos Verlag Berlin. 2004.

Blogger: Richard Watson

For the last 2 months, we have been running a contextual research project, our take on in-depth field research on BPM, as I mentioned here.  It is a fascinating learning experience. Turning the tables on us analysts as we hear stories about how your BPM efforts are progressing is very refreshing.  I couldn’t resist picking out some of the great quotes I've heard to share them here.

Here is a BPM program manager on the activities needed to constantly evangelise in order to get executive buy-in for the program:

"A combination of endlessly and relentlessly communicating about it and demonstrating benefits wherever we could."

The methodology for successful outcomes with BPM is changing, from Taylorist micro-automation, to more collaborative, non-linear, and case management usage patterns.  Using BPM infrastructure at the right level to allow smart people do their work, not get in their way, but get visibility into the results, has been a theme of some of the stories, for example:

"A lot of people treat [BPM] like a production line for a car. If you don't build the car on that production line in the order it needs to go then it's not going to end up the right color for the car the customer ordered.

But the reality is, for example, you see this a lot in insurance, and especially in underwriting – effectively you are doing a sales process, and you are using very knowledgeable people to do the underwriting. If you try to follow a strict process and enforce it you end up actually being able to handle 85% of the work, but it's the low value stuff.  What you try to do then is automate the processes around the high value things, where there's high risk, or huge clients. The moment you try to automate those processes that are really more projects than processes, you end up in big trouble."

This also feeds into customer requirements for BPM tools.  BPM Infrastructure that supports these new ways of work will prosper.  

On the platform side things are also getting interesting, from frustration with products lacking cohesion:

"Anything they are calling a suite is just a grab-bag of their last dozen acquisitions."

Some of the more forward-looking companies are looking for a modular, service oriented architecture to really leverage their existing platform investments:

"The [main] difficultly is to integrate an engine into your platform so you can leverage it. This is something most vendors do not understand…Usually the vendors say, you can import users, import roles, import the organization. Why do you need auditing outside our systems? … In protecting their systems they make integration of the systems so difficult that they don't get leverage from them. In the current market, the leading systems I'm afraid, are architecturally ill.

We will see a 4^th wave of BPM platforms over the next 3-4 years which from the beginning are built for high scalability requirements, support all standards on the market, and different process types. Then you can integrate them fully.  Existing systems do not have a modularity that you need for this."

I told you we were going to challenge the BPM orthodoxy!  In the coming months, I’ll be writing more about these emerging themes: collaborative and social BPM, a modular service-oriented BPM platform architecture and what it takes to get a program going.

If you have BPM experience to share, I'd still be delighted to hear from you.

Blogger: Richard Watson

Clients often ask us for help with their governance processes. We help them design surveys for their development teams to access the maturity of their applications and design practices with regard to sound architecture principles.

Given the problems we in the industry have had producing a formal definition, service oriented architecture (SOA) is best described as a set of design principles that support the goals of building flexible and maintainable systems. The fundamental design principles for SOA are clean separation of concerns, loose coupling, and service-orientation. These principles are not secret sauce to be poured on – if you are building systems that live and breathe these principles – you're doing SOA.

Anne Thomas Manes will publish a report later this quarter detailing the motivations behind, techniques for, and consequences of applying these SOA design principles.

How principled are you?

With any governance processes like these, clearly articulating the motivation for the principles is crucial. No professional, developers especially, like to be told to – just do it – it's good for you. Discovering the impact of abusing the design principles is not difficult to find. Think about these conversations you might have heard in the office.

These are just 3 examples of questions addressed by the SOA principles. If you cannot answer questions like these, you have some work to do in your service portfolio to enforce the principles more cleanly. If we continue to build systems that abuse service design principles, desired outcomes will suffer. If the service granularity is wrong or if interface and implementation concerns are badly separated, then the service will be less reusable. If business logic and infrastructure concerns are blended, developers can't make changes to systems quickly and easily. Tightly coupled systems are going to get in the way of manageability. 

It's the outcomes, stupid

If SOA is about contributing to business outcomes, then measuring the value of services must start with the desired business outcomes. To be credible and repeatable, metrics must be underpinned with measurements from your runtime infrastructure and service repository.  In her forthcoming paper, Anne will frame the principles and techniques with the benefits from applying each principle and with metrics for demonstrating the impact of applying or neglecting each of them. Stay tuned.

Blogger: Anne Thomas Manes

Joe McKendrick posted an article yesterday entitled, "Cloud: the SOA we always wanted, but never had?" He's reporting on a panel discussion he had with Phil Wainewright, Ed Horst, and David Bressler.

Typically I tend to agree with Joe, but not so much this time. I'm more aligned with Mike Kavis on this one.

Mike expressed his distaste for the article in the following Tweet:

I wouldn't go so far as to say that Joe's article is completely wrong. For example, it asserts in the opening paragraphs:

I certainly agree with this point, but the article quickly takes a left turn into hyperbole land when it asserts the following:

• Cloud (as SOA should be) is well understood, and often demanded, by the business
• Cloud (as SOA should be) is platform, language, and technology agnostic
• Cloud (as SOA should) provides greater visibility and transparency to actual IT costs
• Cloud (as SOA should) necessitates binding contracts between service providers and consumers
• Cloud (as SOA should be) is based on trust between service providers and consumers
• Cloud (as SOA should) originates from business requirements

As far as I can tell, cloud computing *is* none of these things. It *should* be. But cloud is too nascent for such assertions. Besides, in order to achieve these characteristics in cloud-based systems, organizations have to 1- design them that way, and 2- develop the contracts and trust described. You won't achieve these characteristics automagically just by deploying a system to EC2, Force.com, or some other cloud provider.

During the second half of 2009, Burton Group is conducting a qualitative research project to assess business, cultural, and technology trends driving business process management (BPM) efforts within the enterprise. We want to understand how organizations are deriving value from BPM. We want to gain insight into the factors that enable them to succeed, and the barriers causing failure. What is the set of product requirements that enable effective and efficient business process design, implementation, integration, execution, monitoring, and management?

A mix of face-to-face and telephone interviews will be the methods used for data collection. Our goal is to have an open-ended dialog with participants from business units and IT groups. The project is open to Burton Group clients and non-clients alike. Some participant roles we are looking to interview include:

· Line-of-business managers

· IT managers, architects and project leaders involved in BPM efforts

· Process and Quality Subject Matter Experts

· Executive sponsors of initiatives related to BPM

We use a research methodology derived from Contextual Design. We conduct a relatively small number (20-25) of lengthy interviews (1-2 hours). Following an interview, we socialize and interpret our notes internally to derive a set of "affinities" -- individual notes, observations, and key insights. After we've completed all the interviews and interpretations, we consolidate the affinities (typically 1000+ individual notes) and look for trends and commonalities.

This effort is being coordinated by Richard Watson, Analyst within Burton Group's Application Platform Strategies service. Information obtained during the interview process will be held in confidence. Any publication or presentation arising from this field research project will not mention any participating enterprise or individual by name without the express written permission of the organization.

Those enterprises involved in the project will have the option to take part in a peer review process prior to publication of the research (e.g., provide comments and factual corrections). Participating organizations will also receive a copy of the resulting research document and have the opportunity to discuss its findings with Burton Group.

This research may challenge perceived industry messaging, but the results will provide deep qualitative information derived by evaluating experiences from multiple customers and finding common patterns.

If you have a question, need additional information, or are interesting in participating, please please leave a comment on this blog post and I will get back to you.

I have been participating in a lively cloud-computing@googlegroups discussion focused on Cloud programming models.

When considering Cloud Computing’s impact on application development and architecture, I have continually asked my team the following questions:

• What is unique?
• What is new?
• How does Cloud Computing change application development and application architecture?

Existing Cloud Computing's momentum is predominantly focused on hardware optimization (IaaS) or delivery of entire applications (SaaS).Building momentum for Platform as a Service (PaaS) has proven trickier because many 'old discarded practices' are being rejuvenated, skepticism abounds, and developers are focused elsewhere. Does current day technology, new products, and accepted culture mean that Rapid Application Development (RAD) environments and Fourth Generation Languages (4GL) will be accepted by mainstream developers? Some proponents think so, others feel like Bill Murray in ‘GroundHog Day’ and wonder ‘why will the concept work this time?’

Interest in Cloud is providing an opportunity to re-think the application platform, review vendor messages (e.g. Google, SalesForce.com, LongJump, Bungee), and determine how application development and application architecture should morph to support Cloud benefits and characteristics.

A recent newsgroup thread discussed how Cloud will impact programming models. A focus on 'programming models' is a red-herring.The best 'programming models' innovations are unseen and instead rely on inversion of control and/or container-based interception.Vendors are extending infrastructure containers and frameworks to transparently support improved dynamic virtualization and more flexible topology distribution (hopefully without new APIs and languages).

But what is new today? As Greg Pfister states in his post, technology concepts such as virtual containers (e.g. Java JVM and PHP interpreters) and decoupling from operating systems and hardware (e.g. java bytecode) have been around for quite some time. Additionally, 'inversion of control' (IoC), 'configurations instead of code' (EJB3.0 annotations instead of java interfaces), and declarative languages (e.g. XAML) are also well established.

But has the infrastructure evolved to transparently realize elastic scalability and optimize resource allocation? or, do developers need to program to the Globus toolkit, .NET WCF, or Force.com APEX?

A straw-man use case is "Can Cloud infrastructure inherently support elastic, scalable, and resource-friendly execution of Plain Old Code Object (e.g. Java POJO, C# class, Tcl script)?" GigaSpaces is a good example of a product enabling more seamless Java code scalability and lower resource footprints when compared to traditional application servers (e.g. IBM WebSphere, Oracle AquaLogic). Another example, code written in C++ which interfaces to the Win32 or LINUX internals is often tightly coupled to hardware, does not following 'service-oriented design principles' (i.e. loose coupling, separation of concerns, interoperability), and may contain single points of contention(e.g. synchronization points, resource locking). The application either doesn't support parallelism, or supports parallelism single machine specific manner.

A Cloud-friendly application supports dynamic deployment of modular components. Picking up and moving a C++/Win32 application to another machine requires moving several operating system resources (e.g. memory, file handles, threads, inter-process communication objects). Decoupling resources behind a service interface is a more Cloud-friendly best practice.

<>A few use case suggestions:

If end-users rely on Software as a Service, then their development environment must support service interactions and corporate infrastructure policies (e.g. security, reliability, availability, performance) should extend beyond their internal data center and span external service providers. WS-Policy based specification standards are a step in the right direction. Policies must be externalized from application code to enable more distributed infrastructure decisions. Following JSR250 and embedding security policy within Java code doesn't facilitate communicating the security policy statements beyond the JVM environment. You will find numerous examples of both .NET attributes and Java annotations which are deployment infrastructure descriptors. Embedding deployment details in code is an inefficient programming model and not Cloud-friendly. The application design technique often breaks the service-oriented concept of 'separation and concerns'.

If end-users rely on IaaS to deploy applications, then having more insight into the application footprint will enable more intelligent resource allocation decisions. Slimming down the application footprint to handle a single unit of work will enable more requests to be packed into a machine.