I would just add a couple points to the discussion.  The first regards an explicit sorting of BPMN 2.0 shapes and symbols into subclasses for the purpose of enabling tool interoperability.  I tried hard to get this into the draft last May but IBM, Oracle, and SAP didn’t want to commit to anything specific at that time regarding their initial “BPMN 2.0″-branded offerings.  Apparently they are feeling more comfortable about it now, as Robert believes there is a chance this could make the final draft.

Even though BPMN defines only 3 basic flow elements – activity, gateway, and event – counting all the variants defined by border style, icons inside, markers, and diagram placement means there are dozens and dozens of semantically distinct shapes.  No tool vendor with an associated BPMN engine, even a simulation engine, is going to support every single one.  And that’s just the part of BPMN above the waterline.  Beneath each element are countless sub-elements and attributes required to define executable detail.

So if you create a model in tool A and expect to import it to tool B, both of those tools have to support a common subset of elements.  That’s what the subclasses are all about.  Here is my view of them.  The Simple class is setting the bar essentially at floor level.  It is meant purely as a lowest common denominator.  If a tool cannot support these shapes and serialize to BPMN 2.0 it should not be allowed to call itself a BPMN 2.0 tool, period.  The Descriptive class corresponds to the Level 1 palette of my book BPMN Method and Style.  It reflects what I believe are the shapes and symbols that should be understood by any business user who wants to read or write BPMN effectively.  It is a challenge for tool vendors because it includes collaboration (pools, message flows, message start and end events).  I hope those things stay in.

What used to be called the Analytical class, corresponding to my Level 2 “core” palette, got renamed DODAF in Robert’s preso.  It is very close to the element set standardized by Michael zur Muehlen in working with the DODAF people.  I agree with Sandy that calling this BPMN class DODAF is maybe not the best for an international standard, but realistically it’s all about user buy-in, and DODAF means a lot there.  I suspect if OMG goes for it they will change the name.  This class includes the most commonly used intermediate events – Message, Timer, Error – plus a few more – Escalation, Signal, Conditional.  And additional gateway types, iteration markers, and other things.  It sets the bar a bit high for tool vendors, but on the other hand, elements outside of the Analytical/DODAF class are unimportant, almost never used, and stand very little chance of ever being interoperable between tools.

OMG has studiously avoided any actual test of compliance for BPMN 2.0, so getting these subclasses into the final spec would be a major leap.  Yes it’s true that most users simply assume that a “standard” implies tool interoperability, but that has been mostly a fake objective in this case.  I think there is hope this time.  I’ve seen a beta of IBM’s Compass BPMN editor and it is very close to the Descriptive level, maybe a bit more.  And I expect Oracle will have something similar.

But you have to understand that these standards are inherently very conservative, and always favor tool vendor “adoption” – even in a proprietary way – over constraints assuring interoperability.  That’s why it’s important for OMG to hear from  users telling them they demand some level of practical tool interoperability, such as this subclass proposal.  If you agree, please email the BPMN group and tell them.  The address is bpmn2-ftf@omg.org.

The second point concerns Robert’s comments on the graphics information in BPMN.  I wrote about this previously (The DI Mess), and in the end even the BPMN team managers in OMG admitted that what they put forward in the draft spec was unworkable.  The new directives were to make the graphics interchange model a real BPMN-specific xsd and eliminate redundancy with the associated semantic model.  Scott Schanel and I put forward a BPMNDI.xsd proposal that does this, and Robert’s preso says that the official OMG effort here carries forward our ideas.  Having seen a draft of it, I would dispute that — it’s still too much like the old one for my taste — but the good news is that there seems to be forward movement at last on it.  More good news is that Denis Gagne is driving the xsd side there  (still the tail on the UML dog unfortunately), and since he is launching the Process Incubator tool referenced by Robert – mappings between Visio, BPMN2.0, and XPDL 2.2 representations – he will ultimately force himself to see which parts of BPMNDI are practical in real tools.  That’s always helpful.

There is a lot still to do on BPMN 2.0 before the final vote, but all of it “under the covers” so to speak.  Nothing significant in the notation or semantics is going to change.

Q1. Exception propagation from subprocess

Q2. Loop and Multi-instance activities

Q3. Wait for requested document or timeout

Q4. Conditional activities

Q5. Alternative responses or timeout

Q6. Timed interval

Q7. Order cancellation

Q8. Missed SLA

Q9. Order change

Q10. Abort concurrent activity

Achieving BPMN Competence

If we go back just a few years, four vendors on the business-centric end of the BPMS landscape stood out by empowering business to play a direct role in process implementation:  Lombardi, Savvion, Fuego, and Appian.  Their software featured model-driven design based on the BPMN standard.  It encouraged a new agile iterative design style based on business-IT collaboration rather than tossing business requirements over the wall.   Where most BPMS vendor projects operated in a bubble disconnected from their customers’ larger business process modeling and analysis efforts, these four vendors stood out by saying it would be better to unite them, to put business at the center of BPMS, not just at the center of process modeling and analysis.  If Smith and Fingar’s 2002 BPM: The Third Wave was the vision, these four vendors came closest to fulfilling it.

Fuego was acquired a few years ago, and now suddently two of the other three are gone, absorbed into the larger SOA/middleware melting pot.  Appian is still hanging out there, but probably not for long.  What does it mean?  Maybe even greater success for BPM under the wing of larger, more financially secure companies.  But I worry that the notion of business-empowered implementation, those BPMS vendors’ sole reason for being, if you ask me, will fade away. 

There is room for hope.  Fuego, absorbed into BEA and now Oracle, today seems to be the tail wagging the Fusion middleware dog, at least the BPM piece of it.  IBM could choose to follow a similar path with Lombardi, although their initial positioning has Lombardi pigeonholed in a separate “departmental” BPM corner.  Progress does not have another BPMS to complicate the Savvion product integration, but the synergistic Progress components featured in the acquisition announcement – complex event processing and “business transaction assurance” - seem quite disconnected from what Savvion brings to the table.

I like Savvion’s products and what they have brought to the BPM market.  They were the first to offer a free BPMN modeler, for instance, and even today they continue to innovate with features like tabular process design and analysis.  If Progress can embrace those business empowerment values and not bury them beneath an IT-centric middleware stack, maybe this will signal a BPM market heading to a brighter and more mature future.

Before describing the integration of decision modeling with BPMN, I need to summarize the Decision Model approach described by Barb von Halle and Larry Goldberg in their excellent book.  I can’t do it justice in a simple post; if you want to find out more, just get the book.

The Decision Model

Although the book does not use the term, I would call it a metamodel for business decisions, i.e., a description of the relationships between specific decision components and the principles governing how they can be combined.  It does not specify a rule language, and is in fact language-independent.  Business-oriented decision models, which are the focus of the book, use a structured form of natural language to define business rules.  Those are easily mapped to IT-maintained data models and engine-specific rule languages for execution.

Just as the relational model imposes constraints on the organization of datasets, the Decision Model imposes constraints on the organization of rulesets.  A business decision is defined as “a conclusion that a business arrives at through business logic and which the business is interested in managing.”  The Decision Model provides the framework for structuring that business logic.  It makes most sense to use the Decision Model and its associated methodology when the decision has high busness value or the decision logic is subject to change, or is complex, or is reused in multiple business processes.  In those situations, the Decision Model’s advantages in management and governance justify the small additional burden its structure imposes.  It is most useful for organizations like financial services or insurance, where the number of decisions that must be simultaneously maintained and tweaked can run into the hundreds or thousands.

Here is an ultra-brief summary of the Decision Model:

A business rule, or business logic statement, relates a set of condition facts to a single conclusion fact.  A fact is a value of an attribute of some business concept, for example, Order.amount or Loan.Risk.  The fact type defines a domain of enumerated fact values used in the decision logic.  For example, the domain of the fact type Loan.Risk could be High, Medium, or Low.  (Technically, the domain could be a continuous numeric or date/time range, but for all intents and purposes it’s easiest to think of it as a set of enumerated values or ranges.)

Within a business rule, the conditions are all ANDed together, i.e., if Condition 1 AND Condition 2 AND Condition 3, then Conclusion.  You may not have OR’ed conditions in a business rule.  The OR’ed condition defines a separate business rule.

The set of rules concerning a single conclusion fact type is called a rule family.  It looks very much like a decision table, with a column for each fact type (conditions plus a single conclusion) and a row for each rule in the rule family.  The book is insistent that a decision table and a rule family are not the same thing, but my view is that people would better understand it if a rule family were described as a decision table subject to certain constraints.

The business logic of a rule family is declarative.  The order of the columns and rows in a rule family is immaterial, as is the algorithm by which it is evaluated on an engine.

A condition fact in a rule family may be the conclusion fact of another rule family, setting up a dependency relationship between the rule families.  Within a single decision model, those relationships must be strictly hierarchical; circular relationships are not allowed.  The top-level rule family represents the decision itself.

For example, the rule family for Loan.Risk could depend on the condition facts Borrower.AbilityToPay and Loan.LoanToValue, each of which would be defined in dependent rule families.  The Borrower.AbilityToPay in turn is based on facts such as Borrower.Income, Borrower.OtherLoans, Borrower.NetWorth, etc.  A condition fact type in a rule family is always either a dependent rule family or simple persistent data.  There is no limit to the nesting of dependent rule families.  Factoring the business logic in this way is the key to business agility, logic reuse, and effective governance.

The Decision Model also specifies a graphical notation for describing the rule family dependencies and a decision table-like structure for specifying a rule family.  That’s it in a nutshell.

Relating the Decision Model to BPMN

Now the question is how does a decision relate to BPMN.  In an appendix chapter in The Decision Model book, I suggested that it should be modeled as a Business Rule task, a new task type in BPMN 2.0.  My thinking when I wrote that last Spring was that beginning modelers mistakenly use gateways in BPMN to model decisions, which are rightly modeled as activities.  But now that I actually understand the decision model (read the book, took the training), I see that suggestion is correct only in certain cases.  The general case is more complicated.

The reason is that a Business Rule task is a single unit of execution.  It models the way a rule engine works, i.e., the condition facts are assumed to be simultaneously available to (or provided to) the task prior to execution, which returns the conclusion fact.  Nothing of consequence happens “in the middle.”

So integrating decisions and process comes down to this:  In a decision model, what is the unit of execution?  How much of it happens “all at once”?  It’s not necessarily the whole business decision.  In the Decision Model, the scope of a single decision is really determined by its business impact, not its implementation.  A decision is a logical abstraction, not a model of how that decision would be implemented in a business process. 

There is still great value in managing business logic across the enterprise at the decision level, i.e., as declarative statements independent of implementation, which could vary or change over time.  But to properly integrate decisions and process, we need to tweak the model in order to accommodate the procedural aspects of decision implementation.  I am thinking of a form of a decision model composed of nodes that map directly to BPMN nodes.  Let’s call it the Decision Implementation Model (DIM).  Nodes in DIM represent decision fragments corresponding to specific  BPMN activity types in the process model.  Thus DIM effectively separates the procedural and declarative aspects of a decision as it is actually implemented.

In DIM, the unit of execution is a rule family.  Here execution does not mean automation, just implementation in the context of a business process.  For purposes of process model integration, we’ll define three rule family types:

1. Executable rule family (ERF).  Business logic is explicit, and all condition facts are available for immediate execution, i.e., either persistent data or the conclusion of another rule family.  In BPMN, ERF is modeled as a Business Rule task, such as implemented by a business rule engine.
2. Human decision (HD).  Business logic is opaque (it’s a human decision), and execution could take time.  In BPMN, HD is modeled as a User task.
3. External decision (ED).  Again business logic is opaque (not necessarily a human decision) and execution is long-running.  In BPMN, it is modeled as a Send/Receive task pair (or equivalent message event pair).  An example might be a credit score provided by an external service. [Note: In BPMN Method and Style, a Service task is interpreted as a synchronous request and short-running, while a Send/Receive pair is interpreted as an asynchronous request and long-running.  But this is really a matter of modeling style.]

Thus to describe decision implementation within a business process, we may need to decompose the decision model into finer-grained DIM rule families.  As in the original decision model, a condition fact in a DIM rule family can be populated by a conclusion fact from a child DIM rule family.  However, in DIM the link from child to parent rule family implies procedural behavior, i.e. sequence flow in BPMN.  [Note: If a rule family is ERF and all its direct dependent families are also ERF, then they could be combined as a single unit of execution (Business Rule task) in BPMN.]

In this segmentation, the difference between HD and ED comes more from BPMN than from the decision model.  In HD the decision is made by a performer within the process, and in ED it is made by an external participant.  In decision model terms, there is little difference between them, as both represent opaque decision logic and long-running execution.  By “opaque,” I mean that the business logic is not a predefined expression of a set of facts.  However, it is possible that an opaque decision (HD or ED) requires certain facts as input, so it could be dependent on other rule families in DIM.

Besides separating the procedural from the declarative portions of the implementation, we need to consider another important element missing from the decision model: actions.  The only process action represented by a DIM rule family is making the decision, i.e. executing the business logic.  It does not include actions such as:

• Retrieving data from its source into the DIM fact domain
• Pre-validating fact data and correcting it if necessary
• Sending notifications to participants and stakeholders
• Updating systems with conclusion fact data
• Handling timeouts and exceptions within the decision implementation
• Executing other conclusion-triggered actions, such as starting another process

All of these actions are represented in the process model as BPMN activities or event handlers.  Except for executing conclusion-triggered actions, they are best represented as nodes within a Decision Subprocess. Each DIM rule family maps to a Decision Subprocess containing one activity representing the DIM rule family type and other activities representing these other actions.  [Note: the BPMN 2.0 specification does not include a Decision Subprocess type.  Its semantics are the same as a regular BPMN subprocess, embedded or reusable.]

Conclusion-triggered actions are typically represented by a gateway following the Decision Subprocess with a gate for each tested conclusion leading to the triggered sequence of actions.  Since rule family conclusion values are exclusive alternatives, the paths can be merged downstream without a gateway.

Decision Model Example

To illustrate how all this works, consider the decision to approve a loan.  The Decision Model, in the Goldberg/von Halle notation, looks like this:

The top-level rule family Loan.ApprovalStatus represents the decision itself.  Rule families are named after their conclusion fact.  The “ticket” shape enclosing the rule family lists its condition facts.  Those above the dotted line are dependent rule families, shown by other tickets in the diagram.  Those below the dotted line are expressions of perstitent data.  Here Borrower.CreditRisk is a dependent rule family and Loan.SpecialConsideration is an expression of persistent data, in this case a human decision.  Borrower.CreditRisk itself dependent on two other rule families.  Factoring the business logic in this hierarchical way enables business agility and logic reuse.

The business logic is apparent from the Decision Model’s tabular rule family notation:

A blank Condition cell means this condition is not used for this particular rule (row).  The Decision Model does not require it, but it is convenient to model conclusion fact types as “abstract” enumerated values like High, Medium, and Low.  Specific numerical ranges are used in expressions of persistent data.  Note in this business decision, all of the logic ultimately depends on just four persistent data facts: Monthly Income, Monthly Payment, Reported  Credit Score, and the Special Consideration human decision.  The rule family structure, however, makes the decision logic easy to parameterize, change easily, and manage.

Mapping the Example to DIM

In the previous example, the Decision Model acts as if all of the source facts are available before executing the decision all at once.  In real business processes, this may not be the case.  Here, for example, we’ll say that the borrower’s monthly income and loan monthly payment facts are data available internally within the process making the decision.  Reported Credit Score is an external decision, made by a credit reporting service.  Special Consideration is a human decision made within the process.  And here we’ll say that this human decision requires IncomeToPmtRatio as an input fact, setting up a dependency on another rule family.

Now ReportedScore and SpecialConsideration are no longer considered persistent data, but decisions, broken out into their own special rule families.  HD and ED rule families here are shown with a dashed border.  The other rule families, shown with the standard solid border, are ERF.  Note IncomeToPmtRatio is a child of both CreditRisk and SpecialConsideration rule families.  This does not violate the no-circularity principle.

Now each of the DIM rule families maps to an activity in the BPMN process model for the decision, as shown below:

We’ll look at adding exception handling and other actions in Part 3.

[Updated Jan 5, 2010, 8:47am PT]

Oh well… The goal was to get people to think about training.  I guess it should.  Click here to take the test.

I thought I would get them all right, but I missed two (tracing through the maze is not so easy).  That put me 99th out of 283 who had taken the test so far, which says a lot by itself.

Anyway, while the test seems totally misdirected as to what is important to know about BPMN, it is a quite clever marketing tool for BPMN training.  There are hints in that direction on the self-test site, although, as I say, I don’t think the test itself would motivate the kind of business process analysts and architects I encounter on this side of the Atlantic. 

So I decided to create my own BPMN self-test, more reflective of the understanding I think is important.  It’s only 10 multiple choice questions, each containing four diagrams, one of which is the correct answer.  I would not be embarrassed to call it marketing for my book and training, and it’s helping me get acquainted with the capabilities of the iLinc environment I am using for my virtual classroom offering. 

Here’s the link to my BPMN self-test.  Unlike the other one, you have to register for mine – basically name, company/title, and email.  It sticks to the core Level 1 and Level 2 palette, but it is reasonably challenging if you don’t know BPMN well. 

Try them both, and tell me what you think.

My take is this was bound to happen.  I’m sure Lombardi has rebuffed any number of BPM suitors over the years, insisting on an eventual IPO.  But in today’s market that exit must have looked farther away than ever, so Phil Gilbert and company could forget all about the past Evil Empire bashing and just take the money (amount undisclosed).

The functional overlap between the Lombardi and IBM BPMSs is obvious.  I am not optimistic about the positioning, and I doubt they have even figured it out yet.  Who can forget the business about “content-centric” vs “process-centric” BPM at the time of the FileNet acquisition?  In the slide deck IBM sent out this morning, a couple graphics seem to be following that same kind of thinking.

Here, for example, IBM is BPM for the Commercial LOB, the Financial Services arm, the Agent network, etc.  Lombardi is BPM for one branch of the Commercial LOB, actually for the little circles (departments?) hanging off it.  Enterprise vs Departmental.  It says it right there on the slide.

The next one is even worse.

BPM involves process (Websphere), information (FileNet), and people (Lombardi).  Now we have a separate BPMS for each of those.  Isn’t that great?  Ummm, no.

OK, to be fair, I’m going to assume the IBM marketing guys found out about this a couple days ago, and this is what they could throw together for the announcement. 

Usually IBM is very slow to integrate its BPM-related acquisitions, e.g. FileNet or ILOG.  They just add to the list of stuff that the professional services guys can integrate for you.  Webify was the counter-example.  Renamed WebSphere Business Services Fabric, it was just jammed into the middle of the WebSphere BPMS, even though it retained a completely separate metamodel, vocabulary, and toolset.  It confused the heck out of everybody.

I think IBM would be better off adopting a strategy similar to Oracle’s when they bought BEA.  Again, two BPMSs, one more human-centric and the other more integration-centric.  But instead of just adding to the list of products you could buy, Oracle aggressively went after technical integration, melding BEA’s vastly more business-friendly tooling with Oracle’s strength on the runtime backend.

There is no doubt IBM has been trying to make its own BPMS tooling more like Lombardi for a year now.  Business Space and some of the human task features in v6.2 and v7 are a testament to this.  But they could never really grab hold of Lombardi’s secret sauce, the “shared model.”  What you model is what you execute, instant playback from the design environment, agile iterative design and deploy…  WebSphere BPMS just doesn’t work that way.

Positioning that approach as “departmental” is a losing strategy, and I honestly doubt that in the end IBM will do that.  The battle for enterprise BPM is about engaging the business at enterprise scale, not just IT.  It’s what Phil Gilbert has been preaching for years, and I don’t think IBM would have coughed up the money if they didn’t believe it, too.  So the trick is melding Lombardi’s superior business-friendly tooling with IBM’s bulletproof backend and SOA.  BPMN 2.0 will provide them a way to do that, so I think today’s positioning slides are just placeholders. 

But then again, they didn’t ask me.

Today, both disciplines share the goal of empowering business by allowing models to be created and maintained by business analysts for the purpose of documentation, analysis, and governance.  And both allow those business-oriented models – if the organization so chooses – to be transformed into technical models, executable on a runtime engine.  In recent years, tremendous progress has been made within each domain toward standards and methodologies that enable a common modeling language to be shared by business and IT, i.e. roundtripping between the business and technical models. In the process domain, such a structured approach is exemplified by my book BPMN Method and Style.  In the BDM domain, The Decision Model by Larry Goldberg and Barb von Halle applies a similar structured business-oriented methodology.

Neither book, however, gives sufficient consideration to the fact that in decision-intensive processes like lending or claims, process and decision modeling are separate large-scale activities performed concurrently, usually by independent teams.  That can work well, but only if the methodologies on both the process and decision sides explicitly consider how those models will be integrated. 

Even though both books are brand new, neither discusses this. That does not mean the books are incorrect.  Each just describes a core foundation, a framework for modeling business processes or business decisions so they can be shared, analyzed, and maintained effectively.  But creating process models and decision models that fit together – in a way that serves both the analyst/architect concerns of documentation, analysis, and governance, and the developer’s concerns of executable implementation – requires going a step further.  Larry, Barb, and I are beginning to do that now.

Both BPMN Method and Style and The Decision Model properly give a lot of attention to the question of cardinality: What is one process, as opposed to multiple related processes?  What is one decision, and how does it relate to its constituent business rules?  Both frameworks impose some technical constraints on it, but beyond that, the scope of a single process or a single decision is a matter of judgment or art, what I call modeling style.  Both a process and a decision, the top-level objects in each domain, are inherently compound entities, composed hierarchically out of reusable building blocks.  In BPMN, those building blocks are called subprocesses; in the Decision Model, they are called rule families.  A BPMN subprocess can call other subprocesses, and a Decision Model rule family can call other rule families.

Integrating BPMN and decision modeling ultimately comes down to properly representing decisions and their constituent rule families in the context of BPMN subprocesses and tasks.  There is a way to do this, and we’ll cover it in Part 2.

They begin with the refreshingly honest statement (with which I concur 100%):

A notorious problem is the low level of modeling competence that many casual modelers in process documentation projects have.

On the other hand, the authors’ assumed tool for these folks is ARIS or  Casewise, which are really tools for architects not casual modelers.  The diagrams in the paper are all EPC.  I wonder why they did not write the paper in context of BPMN.  But let’s look at the recommendations.  After applying some academic hocus-pocus, they arrive at the following prioritized list:

1. Make the model as structured as possible.  That’s one of my top ones also, but they mean something completely different than I do.  To Mendling et al, “structured” means “block-structured.”  That would make Michael Rowley happy, but not mainstream BPMN modelers.  No, I think these guys are living in a BPEL yesterworld.  Enforcing block struture probably does reduce syntactical errors, but at a heavy price: Only developers are willing to go along with it.  In BPMN Method and Style, I emphasize structured models as well, but I mean hierarchical structure, in which a single end-to-end semantic model can be viewed at different levels of detail.
2. Decompose a model with more than 50 elements.  I’m not sure what constitutes an element, since EPC diagrams include everything but the kitchen sink.  In BPMN, my recommended style limits you to 5-10 activities in a process level (subprocess). 
3. Use as few elements in the model as possible.  In other words, if an element is redundant, leave it out.  I agree with that.  But how did it make the top 3?  Kind of a motherhood statement.
4. User Verb-Object activity labels.  Yes I have that one too.
5. Minimize the routing paths per element.  I agree that expressing process variation by overuse of gateways and alternative paths makes models harder to read.  But minimizing routing paths as a broad guideline seems misguided.  The model should reflect how the process works, first and foremost.  I think the most likely response to 7PMG’s recommendation would be to omit exception paths from the diagram entirely.  Which would be a bad thing.
6. Use one start and one end event.  I’m on board with one start event, but definitely NOT one end event.  Using separate end events to represent distinct end states of a process or subprocess is critical in my method and style.  It’s what makes hierarchical diagrams traceable from the top down.
7. Avoid OR routing elements.  Again the BPEL/block bias.  But it would seem to contradict items 3 and 5 above if the process semantics really are conditionally parallel flow.

On the whole, I don’t agree with the conclusions.  But at least they are asking the most important question.  What do you think?