Richard Veryard on SOA

Is Enterprise Architecture a Science?

2009-07-09T20:02:00.004+01:00

asks @RSessions (Roger Sessions)

I have long argued that the answer is no. What passes for knowledge in enterprise architecture is largely a combination of anecdote and received opinion. Intellectual effort is devoted to hierarchical forms and elaborate classification schemas, based on abstract reason rather than empirical measurement; this kind of work looks more like mediaeval scholastic philosophy than modern science.

In comparison, the followers of Christopher Alexander seem like nineteenth century gentleman scientists, carefully collecting specimens from which they try to infer useful principles and patterns. Alexander's four-volume masterpiece on the Nature of Order is a brilliant and fascinating work, which I think every enterprise architect should read (in their spare time), but it's not exactly suitable as an everyday handbook.

In order for enterprise architecture to qualify as a science, it has to follow scientific method. Now I am pretty broadminded about what counts as scientific method, but it's more than mere predictability. (Card games are predictable, but that doesn't make cribbage a science. Roger says if card games were predictable, he'd be a rich man. But if card games were not predictable, the casinos would go broke.)

Anyway, you can get predictability from art. I saw Dave Crosby on a documentary once, criticizing the fact that the Eagles' concerts were note-for-note predictable, CSNY preferring a slightly looser style in response to each audience. So what if EA's an art, asks @HotFusionMan (Al Chou).

Peirce understood the limitation of scientific method, holding "that slow and stumbling ratiocination can be dangerously inferior to instinct, sentiment, and tradition in practical matters" (Wikipedia).

So does it matter if (as I believe) Enterprise Architecture is not a science? The key question that raises is - what is the source and status of EA knowledge, and how can we ever resolve matters of opinion, except by obscure mediaeval argument?

SOA Retrospective

2009-06-30T19:17:00.004+01:00

As this is my last day as a full-time employee of the CBDI Forum, I thought I'd permit myself a little retrospective discussion on Service Oriented Architecture.

My own understanding of the "service" concept can be traced back to a number of things I was doing in the 1990s, including enterprise modelling for open distributed processing (ODP), as well as early structured methods for component-based development (CBDI or CBSE). From this perspective, it wasn't hard for me to see that the service concept represented a significant departure from the software paradigms I had learned at university (everything from SIMULA to PROLOG).

By the late 1990s, the software vendors were pushing component-based development. It was already obvious to some of us that the most important thing about a component was not the internal mechanism (its implementation) but the service it delivered. I started attending meetings of the CBDI Forum, and writing for the CBDI Journal. My main focus however was not software but business - developing a business modelling approach that would link component-based software with component-based business transformation. I developed a component methodology called SCIPIO, and in 2001 I published a book on the Component-Based Business, which tried to explain business architecture in terms of business components - coherent chunks of business capability, interacting through service relationships.

But although my friends at CBDI Forum understood this, we found that a lot of people in the software industry still saw components merely as reusable lumps of software - a rehash of modular programming or object-oriented programming. What was often missing from this story was any sense of architecture. However, even at that time, a few methodologies did take architecture seriously: there were some good ideas (including explicit recognition of the service concept) in Select Perspective, for example.

By the early 2000s, people had started to talk about service-orientation and service-oriented architecture. Gradually the large software vendors got their hands on these concepts, and started to sell products and platforms into a growing market called "SOA". Lots of people started to equate SOA with these products and platforms. Meanwhile, some SOA evangelists started to make sweeping (and usually ungrounded) promises about business agility and transformation. A gap started to emerge between expectation and reality.

Some of us had experienced something similar before. In the 1980s, we had worked with Information Engineering and CASE tools, which were sold on the promise of transforming IT productivity. However, achieving these outcomes generally depended very little on the features of the technology and much more on how the technology was implemented, used and managed. I developed a technology change management roadmap, which JMA (and later Texas Instruments) used for planning the adoption of its methodology and technology into some of its larger customers.

So many years later, CBDI developed an SOA Roadmap, along similar lines, in order to help large customers adopt and exploit SOA. And in 2006, I joined CBDI as a full-time employee. One of my tasks was to help turn a considerable body of knowledge (as published over many years in the CBDI Journal) into a well-structured methodology (known as SAE - Service Architecture and Engineering).

Although my instinctive sympathies were always with the evangelists who talked about what kind of future business transformations might be possible by extrapolating the available technologies, I started to get impatient with their inability to talk concretely about business change. At the other extreme, I was underwhelmed by some of the rather narrow and uninteresting uses to which SOA technologies were sometimes being put. Regular readers of this blog will have seen me writing frequently on these subjects.

One source of growing complexity, however, is that SOA can no longer be regarded (if it ever was) as an isolated discipline. On the one hand, it tends to be combined with all sorts of other technologies and practices, including BPM, Business Intelligence, Event Processing (Complex or Otherwise), Grid, Cloud, and anything 2.o. On the other hand, it tends to be allied (sometimes awkwardly) with other architectural disciplines, notably Enterprise Architecture. And there are usually major business changes going on, such as Merger and Acquisition. Whether you are doing a technology change roadmap or a business case, you probably need to juggle several of these factors. In many situations, SOA is no longer the rising star of technology adoption but merely one of the supporting acts. David Sprott, the founder of the CBDI Forum, now sees service architecture as business as usual (BAU), saying that "the key questions to be answered revolve around the level of standardization, componentization and rationalization in business and IT" (SOA in the Recession, Feb 2009). Some SOA pundits have even suggested that SOA (or at least the label "SOA") is dead. See my post Has SOA gone for a Burton?

I have no doubt that the CBDI Forum will quietly continue applying the concepts and principles of SOA to practical business problems, unbothered by all this hype. I remain more interested in business transformation than in technology for its own sake, but I shall continue to blog here from time to time on SOA and related topics. For other topics relating to systems thinking for innovation and business survival, you may want to subscribe to my Demanding Change blog.

Deconstructing The Grammar of Business

2009-06-18T22:45:00.002+01:00

@JohnIMM (John Owens) trots out a familiar piece of advice about data modelling today.

"Want to know what data entities your business needs? Start with the nouns in the business function names."

Starting with the nouns is a very old procedure. I can remember sitting through courses where the first exercise was to underline the nouns in a textual description of some business process. So when I started teaching data modelling, I decided to make this procedure more interesting. I took an extract from George Orwell's essay on Hop-Picking, and got the students to underline the nouns. Then we worked out what these nouns actually signified. For example, some of them were numbers and units of measure, some of them were instances, and some of them were reifications. (I'll explain shortly what I mean by reification.) Only a minority of the nouns in this passage passed muster as data entities. Another feature of the extract was that it used a lot of relatively unfamiliar terms - few of us had experience measuring things in bushels, for example - and I was able to show how this analytical technique provided a way of getting into the unfamiliar terminology of a new business area. I included this example in my first book, Pragmatic Data Analysis, published in 1984 and long out of print.

One problem with using this procedure in a training class is that it gives a false impression of what modelling is all about. Modelling is not about translating a clear written description into a clear diagrammatic structure; in the real world you don't have George Orwell doing your observation and writing up your interview notes for you.

Now let me come on to the problem of reification. The Zachman camp has started to use this word (in my view incorrectly) as an synonym of realisation - in other words, the translation and transformation of Ideas into Reality. (They can perhaps find some support for this usage in the work of the Arab philosopher ibn Arabi, who talks about entification in apparently this sense.) However, modern philosophers of language use the word "reification" to refer the elevation of abstract ideas (such as qualities) to Thingness. One of the earliest critics of reification was Ockham, who objected to the mediaeval habit of multiplying abstract ideas and reified universals; his principle of simplicity is now known as Ockham's Razor.

In our time, Quine showed how apparently innocent concepts often contained hidden reification, and my own approach to information modelling has been strongly influenced by Quine. For example, I am wary of taking "customer" as a simple concept, and prefer to deconstruct it into a bundle of bits of intentionality and behaviour and other stuff. (See my post on Customer Orientation.) As for business concepts like "competitor" or "prospect", I generally regard these these as reifications resulting from business intelligence.

Reification tends to obscure the construction processes - tempting us to fall into the fallacy of regarding the reifications as if they directly reflected some real world entities. (See my posts on Responding to Uncertainty 1 and 2.) So I like to talk about ratification as a counterbalance to reification - making the construction process explicit.

Of course, John Owens is right insofar as the grammar of the data model should match the grammar of the process model. And of course for service-oriented modelling, the grammar of the capabilities must match that of the core business services. But what is the grammar of the business itself? Merely going along with the existing nouns and verbs may leave us short of discovering the deep structural patterns.

A Value Proposition for Enterprise Architecture

2009-06-15T15:30:00.004+01:00

#EAC2009 Something else I took away from Sally Bean’s and Peter Haine’s workshop Reflecting on EA at EAC 2009 was the relationship between two sets of questions.

What is EA all about, what is the (emerging, changing) identity of EA?
What is the value proposition for EA?

I personally find the second of these two questions much more important and interesting than the first. Questions of identity often result in entrenched positions, and (as I discussed in my previous post Think Globally, Act Locally) can produce division between different views. In the case of Enterprise Architecture, there is a traditional view (EA-as-IT-planning) and an emerging view (EA-as-business-strategy).

I think the question about the value proposition opens up a much more interesting discussion about the possible evolution and potential multi-tasking of enterprise architecture teams.

(EA itself needs to understand its business model to help it survive. The full exploration of the value proposition needs something like the Osterwalder business model canvas we discussed in our workshop on on Business Modelling for Business Improvement, so I'll have a go at that. See below for Osterwalder template.)

One of the key questions for the value proposition is the timescale. Sometimes enterprise architecture is described in terms of longer-term value: through-life coordination and capability management. Some people are still comfortable with this; but other people see a difficulty with the fact that business needs to wait so long to realise this kind of value, or even to measure it properly. In contrast, there is growing support for a much shorter-term delivery of value.

Essentially, that means EA must deliver value within same timescale as projects. And what is the nature of this value? Roger Sessions points to the massive failure rate in IT projects, and argues that's something EA can and should be fixing. In other words, the EA value proposition can be defined in terms of improving IT project success ratios.

I see two difficulties with this. The first is one of perspective. If EA is working closely with the projects, then how is the EA perspective any different from project perspective? And if the problem is that projects are doing things wrong, then how can EA fix the problem from within the project perspective? The EA view of business requirements is hardly going to be very different from that of the good business analyst on the project. If EA is no longer taking the long view, then its value proposition is largely based on the hypothesis that the architects may have a bit more knowledge and experience than the business analysts, and some slightly superior tools and techniques. But we might achieve this outcome more efficiently and effectively by simply upgrading the business analysis practice and redeploying the architects as senior business analysts. Indeed, some IT organizations seem to be moving in this direction, although they haven't taken away the formal job titles yet.

The second difficulty is that the job of overseeing projects and ensuring project success is hugely duplicated. Within a large IT organization, we might have project management, programme management, IT governance, tools and methods, quality management (control and/or assurance) as well as enterprise architecture, each with its own "body of knowledge", each trying to prevent projects from getting things wrong (and claim the credit). The word "silo" springs to mind here. (All of those roles might possibly be held by the same person, but does that remove the complexity?)

From a systems-thinking perspective, this looks completely crazy. If the value proposition for EA is simply to correct things that projects are doing wrong, then this counts as "failure demand". If the value proposition for EA is to make sure that projects are successful, then that's putting the responsibility in the wrong place. It is the project's job to be outstandingly successful. If they can achieve this unaided, this appears to make EA redundant.

In summary, I'm not convinced that the traditional value proposition for enterprise architecture is convincing to its customers, whoever they are. (Who are the customers anyway, the CIO or CFO who have to pay for it, or the business line management and IT project managers who are being asked to spend time and effort on EA "for their own good", and are not always grateful for EA attention?) I think the top priority for the enterprise architecture discipline is to find and formulate a viable and meaningful value proposition. And it really doesn't matter whether we call it "enterprise architecture" or not.

Thanks to several Twitter friends for today's discussion: A Jangbrand, Anders Østergaard, Andrew Townley, Brenda Michelson, Colin Beverage, Roger Sessions, Todd Biske. (Did I miss anyone?)

Business Model Template

View more OpenOffice presentations from Alexander Osterwalder.

EA: Think Globally, Act Locally

2009-06-12T20:39:00.004+01:00

#eac2009 The IRM Enterprise Architecture conference in London this month continued some of the themes of the Open Group Enterprise Architecture conference in April, and I saw many of the same faces.

One of the main themes was the scope of Enterprise Architecture. Many people argued strongly that EA was not just about IT but about Business, and this came across from Chris Pott's keynote.

Not everyone agreed, however. When Roger Sessions challenged Chris from the floor, putting the case for the continued relevance of IT to the enterprise architect, there was a ripple of assent around the audience.

EA undoubtedly has a credibility problem if it aspires to sort out broader problems of "the business" when it is currently perceived to have had limited success with its original remit - the narrower problems of IT planning.

Roger posted the question on Twitter: "Two enterprise architecture camps: focus on the global vs. focus on the IT. Which are you?" Most of the replies took the side of the global. Nigel Green linked to something he had posted to his blog last year (What is an enterprise architect?) and said "I'm the 'global' if you mean Business Technology a la Forrester ... EA is about biz transformation not just IT". Anders Østergaard Jensen said "EA = S + B + T from which we can infer that EA is global", and Colin Wheeler said "I think Enterprise Architecture is a logical framework in which the business can make rational decisions. Definitely part of the global focus for me".

It was Tom Graves who found a way of reconciling both positions, by quoting Patrick Geddes' slogan Think Global, Act Local. "Global. IT alone is too narrow ... Lack of whole-org EA creates problems for IT. Act local (IT) think global (EA). Apply EA in detail for IT-systems etc, but always remember the global." With most of the others, Tom remains convinced about the importance of the global. "EA is architecture of enterprise, not IT - IT is just an implementation, nothing more - drop the IT-centrism!!".

Roger suggested a compromise. "You can't deliver high value IT unless you know how IT relates to the organization as a whole. The role of enterprise architecture is to bring a holistic perspective to IT." Or perhaps ""The role of EA is to bring a holistic perspective to IT-supported business capability."(miket0181)

There is clearly a wide gap between AS-IS (enterprise architects frustrated within the IT department) and TO-BE (enterprise architects respected across the business). Even if we go along with Chris Potts's line that enterprise architecture is a form of corporate strategy, there's a way to go in most organizations. Nothing wrong with thinking about the future, but some enterprise architects have got a day job as well.

EA Archetypes

2009-06-09T09:33:00.006+01:00

#EAC2009 Following my workshop on Business Modelling for Business Improvement at EAC 2009, I caught the end of Sally Bean’s and Peter Haine’s workshop Reflecting on EA.

An interesting discussion on EA archetypes. We talked about the contrast between EA-as-visionary and EA-as-realist. The EA-as-visionary is an optimist who produces value by creating new opportunities and producing economies of scale and scope; the EA-as-realist earns his/her keep largely by stopping ill-conceived initiatives, saying No to pushy vendors, and producing economies of governance. (In January 2006, I put the case for Realism in a debate on Optimism with Jeff Schneider.)

Roger Sessions mentioned an interesting correlation in the US government space between IT failure and Sarbanes-Oxley-driven "investment" in Enterprise Architecture, suggesting that a mere formal requirement to produce EA deliverables may actually destroy value. (Roger discussed this in his recent editorial on Obama's Information Technology Priority; he is planning to include some graphs in his talk tomorrow afternoon.) This indicates a third archetype: EA-as-formalist, bureaucrats playing Zachman bingo with little vision or practical realism.

And yet there is probably a place for formal rigour, if it can be balanced with vision and realism. It is the formal rigour that confers some authority on the architect to promote either vision or realism or both. So how do we combine the three archetypes: Visionary, Realist, Formalist?

While it would be crazy for me to equate this triad with Lacan’s triad (Imaginary, Real, Symbolic), I think there may be some weak structural parallels. Here are some starting thoughts.

Imaginary - For Lacan, this is about constructing coherent images. The EA-as-visionary must be good at joined-up-thinking, and good at story-telling.

Symbolic - For Lacan, this is about representing the images in some language. The EA-as-formalist must be able to create robust representations.

Real - For Lacan, the Real is what resists representation. The EA-as-realist must understand the limitations of both the vision and the formal models.

I don't know how fruitful this parallel is going to be, so I need to think about it a bit more.

What's Wrong with Layered Service Models?

2009-05-25T22:14:00.003+01:00

@JohanDenHaan pointed me at a couple of articles by Bill Poole

Bill takes a particular layered service model, a plausible combination of layers such as you might find in any number of popular SOA books, and shows how he can use this model to produce an inefficient and inflexible design.

Of course, all this shows is that there are problems with a particular layered service model, as interpreted by Bill. (The authors of the books from which Bill has taken this model might claim that Bill had misunderstood their thinking, but whose fault is that?) It doesn't show that all layered service models are bad.

As Bill points out, one reason commonly cited in support of the layered service model approach is the hope that services will be highly reusable. But there is a much more important reason for layering - based on the expectation that each layer has a different characteristic rate of change. (This principle is known as pace layering.)

When done properly, layering should make things more flexible. When done badly (as in Bill's example) then layering can have the opposite effect.

One of the problems is that the people inventing these layered service models often confuse classification with layering. We can identify lots of different types of service, therefore each type must go into a separate layer. A deeper problem with these models is that their creation is based purely on clever thinking rather than systematic and empirical comparison of alternatives. Too much architectural opinion is based on "here's a structural pattern that seems to make sense" rather than "here's a structural pattern that has been demonstrated to produce these effects".

See my post on Layering Principles.

You can't just take an SOA principle ("loose coupling is good", "layering is good"), apply it indiscriminately and expect SOA magic to occur.

Customer Orientation

2009-05-15T09:17:00.008+01:00

@adamshostack objects to something I quoted from Clayton Christensen: Understanding your customer isn't enough (hattip Chris Lawer, Gunnar Peterson).

What Christensen is saying, and I absolutely agree with this, is that understanding the customer is the wrong level of analysis (granularity). What we need to focus on is the job the customers are trying to get done when they use your product or service.

Adam thinks this is "fascinating & wrong. W/o understanding customer orientation, you can't from job" (via Twitter).

I think pharmaceutical sales and marketing provides an excellent example of why Christensen is correct. As I have explained before, a typical twentieth-century business model involved drug company representatives making personal visits to doctors. To support this kind of model, you collected lots of information about the doctor - not just professional (size of practice, specialization, and so on) but also personal (ethnic group, sexual preference, ages of children, golf or squash). The drug company employed a range of representatives of different types, and selected the appropriate rep to visit a white gay squash-playing doctor.

The trouble with this business model is that it is not aligned with the products and services of the drug company. Doctors increasingly regard these kind of sales visits as a complete waste of time; even if they accept the hospitality of the drug companies, they have learned to be resistant to the sales messages.

What the drug company needs to focus on is how to provide more value to the doctor. For this purpose, you don't need information about the doctor, you need information about what the doctor actually does. In particular, you have to understand the decision process in which the doctor thinks about prescribing particular drugs to a particular patient, as well as the collaborative process in which the doctor and other healthcare practitioners discuss alternative courses of treatment with a patient.

Understanding the doctor is missing the point. The opportunity to create business value comes from understanding the work of the doctor. Different doctors may have different styles and habits, and may approach similar cases in different ways (although this is increasingly constrained by procedure and protocol imposed by health authorities or health insurance companies). That's the right level of analysis.

One technique we use for this analysis is business process modelling. But we're not interested in the company's own processes, we're interested in the customers' processes, and in the variations in these processes. Business survival depends on providing products and services that add value to these customer processes, so it's the customer processes we need to understand. Not the customer as a passive and indivisible entity (as in many CRM systems) but as an active bundle of behaviour and capability and purpose.

Semi-structured processes 2

2009-05-10T10:33:00.002+01:00

@jonerp (Jon Reed) kindly sends me a link to an article on the SAP community network website, Value Scenarios and the BPM continuum, discussing the requirement for semi-structured processes.

The author of the article, one Dan Woods of Evolved Media, places "processes automated and supported by BPM" at the "loosely-structured" point of the continuum, but he also talks about "more advanced and structured process design using BPMN and other such techniques"; I don't fully understand his taxonomy of processes, but he certainly seems to understand the need to support semi-structured processes as well as the fully structured processes described by Ann Rosenberg.

Dan's starting point is the SAP book written by Ann Rosenberg and others, but his own analysis goes a lot further. He picks up on a reference to the Geoffrey Moore core/context distinction (which CBDI Forum members will be familiar with), and takes this distinction to its logical conclusion.

Dan also makes clear his opinion that SAP doesn't currently support this continuum. "SAP's integrated story about BPM, SOA, and Value Scenarios will have to be extended to include more unstructured collaboration." While Dan is not an official spokesman for SAP, the fact that opinion article is published on an SAP website does give it some credibility, don't you think?

From Business Intelligence to Organizational Intelligence

2009-05-08T22:30:00.006+01:00

#orgintelligence ...

Business Intelligence (BI) is one component of what I'm calling Organizational Intelligence, so let's look at some useful trends within the BI world that are relevant to Org Intelligence, as well as some criticisms of the current state of the art in BI.

Real-time, event-driven process-driven BI

Over the past couple of years, we've seen the emergence of real-time BI platforms such as SeeWhy. Champions of realtime BI suggest that it can address weakness of conventional BI, including

out-of-date information
failure to identify process problems
lack of suitability as predictors

[source: Three Weaknesses in Business Intelligence Today, CIO magazine, October 2007)

Last month Charles Nicholls (SeeWhy CEO) attacked IBM in his blog for (as he claims) neglecting the web: What about the web, IBM? At eBizQ, Michael Dortch offers a more balanced view: Some Shafts of Light. The real proof will be if IBM's new Business Analytics and Optimization Services unit will address the kind of real-time process issues identified by SeeWhy a couple of years ago.

Nari Kannan also criticizes conventional BI for its lack of connection to the business process. Drawbacks of Business Intelligence Tools When Handling Business Processes.

"When it comes to Process or Operational Intelligence, the problem becomes slightly different. Post mortem, rear view mirror kind of information is not very useful. Knowing that you screwed up a patient's surgery experience or the Automobile Insurance Claim, after the fact is not as useful as getting real-time information as and when it is happening and you have a chance to set things right."

Collaborative decision-making

In January, Gartner published some forecasts about the BI market for the next three years, including this.

In 2009, collaborative decision making will emerge as a new product category that combines social software with BI Platform capabilities. The emergence of social software presents an opportunity for savvy IT leaders to exploit the groundswell of interest in informal collaboration. Instead of promoting a formal, top-down decision-making initiative, these IT leaders will tap people's natural inclination to use social software to collaborate and make decisions.

Dave Linthicum agrees.

"This is occurring now, and is a huge area of growth in BI, if you ask me. With all of that good data being placed on the social networks these days, the BI applications around mining that data is going to be extremely valuable."

Collaborative BI

I predicted the emergence of collaborative business intelligence nearly four years ago (Service-Oriented Business Intelligence). So I am very happy to see early signs that this might now be moving into the realm of practical application. If you are a BI toolmaker or advanced user, I'd be delighted to talk to you.

Will Libraries Survive?

2009-05-02T11:37:00.010+01:00

Following my post on Library Collaboration, Anders Jangbrand commented

"Book available on net do not have same limitation. No phys copies. Will libraries survive?"

That's an excellent question for two reasons. Anders always asks good questions, but another reason I like this one is because I am already working on an answer. But first a little history.

In the dim and distant past when I first learned data modelling, one of the standard exercises in books and training courses was to model a library. These were of course models of the library as an information processing system. Before computing, libraries were managed with cabinets full of hand-written cards. There would be one or more cabinets containing the library catalog, there would be cabinets for members' names and addresses, outstanding loans, reservations, books awaiting repair, and so on.

So it was an apparently straightforward task to model this lot; but there are some hidden traps for the beginner. For example, BOOK sometimes means the book title (as when you reserve a book) and sometimes means the physical copy (which is what you borrow and return). When I teach data modelling, I generally give students the freedom to fall into these traps so I can show them how to get out again.

(Some libraries identify each physical copy individually, while other libraries merely count the physical copies of a given book title. The physical copies are distinct in the real world, but may be indistinguishable in the library's card or computer systems. Information strategy includes making this kind of choice. See Business Concepts and Business Types to see how this is handled in the CBDI SAE method.)

The great advantage of using a library as a teaching example was that most people had used libraries and had some idea how they operate. However, some people started to think that the example was a bit old-fashioned, so they developed a Video Rental example instead. Video rental has pretty much the same information processing structure as a library, so all they needed to do was take the library example and change some of the words on the diagrams. And nothing much changed when video tapes were replaced by DVDs, although there were some minor complications if you wanted to handle both tapes and discs during the transition period.

The traditional library is indeed threatened, but it will probably outlive video rental, which is threatened by much the same forces to an even greater extent. If we want to manage these forces, we need to move away from an information processing model onto a different kind of model.

Libraries and video rental operate a very similar information processing model. But if we want to think about the survival of libraries and video rental, we need a model that shows how libraries and video rental are different from one another, and to what extent libraries or video rental can play a valuable role in the service economy of the future.

The first point to note here is that the book plays a much more varied role in people's lives than the video, even today. The vast majority of DVDs are either feature films or collections of TV programmes, consumed as entertainment, and video rental essentially caters for this market. Books are also consumed for entertainment, but they are also used for reference, study, research and other purposes, and most libraries support a broad range of purposes. If we want to forge a sustainable role for the library of the future, we need to engage with these purposes, and possibly explore some newly emerging purposes as well.

Some people may argue that the business model should be technologically neutral. Anything you can do with a book, you can do with a DVD as well. For example, if you want to learn Spanish, it shouldn't matter whether you borrow a book from the library or rent a DVD. But until video rental gears itself up to support this kind of market, there will continue to be a real difference in the business model between books and DVDs.

The next point to note is that we can open up the idea of the consumer. A lot of people like to read books in groups. The whole group reads the same book (see ambiguity of BOOK above), and then the members meet once a month to discuss the books. This represents an interesting opportunity for the librarian - to provide services to the whole reading group rather than to individual readers. For example, if the library has a dozen copies of a recent novel, then this novel can be offered to the local reading group for next month's meeting.

Now suppose we have a network of a couple hundred libraries around the region, supporting a thousand reading groups between them. Recent novels can circulate around the libraries in sets, to support the needs and interests of the reading groups.

See what we've done here. All sorts of interesting business opportunities emerge by these two conceptual shifts. Firstly changing the concept of READER from individual to reading group (and creating a new composite service for the whole group). Secondly changing the concept of LIBRARY from single library to a network of libraries (and creating new kinds of collaborative process).

Once we've identified these conceptual shifts, we can go back into the information processing model to work out the practical logistics. But the point I'm making here is that conceptual shifts of this kind (and the business opportunities that are associated with them) don't appear unless you step outside the information processing perspective and look at the business through a different lens.

Of course I haven't quite answered Anders' question yet. But I'm working towards an answer ...

Library Collaboration

2009-05-01T18:39:00.006+01:00

University libraries typically have a major resource scheduling problem, as I mentioned in my post on Collaboration and Context.

"Let's suppose that in the first week of February, sixty students suddenly want to read Kant. The university library possesses twenty copies of Kant's Critique of Pure Reason, so most of the students will have to wait or share. By the end of February, nobody wants to read Kant any more, and the twenty copies sit idle on the shelf until the same time the following year."

My first impractical idea for solving this problem was to get collaboration between the library and the teaching staff. Would it be possible to coordinate the teaching, so the students don't all need the same books at the same time? (Anyone who has worked in a university will know why I reckon this is impractical.)

My second idea might be a little more practical: to get collaboration between university libraries. Perhaps fifteen universities can share two hundred copies of Kant and ship them around as required. Possibly not as much scope for efficiency as my first idea, but it may be easier to implement, as it only involves librarians cooperating with other librarians rather than with professors.

However, the most plausible idea for solving the problem is to involve the paying customers - the students. In many countries higher education is subsidized by the state, but students and their parents are now being asked to pay a much higher contribution, and this makes them stakeholders in the economics of the university and the quality of its services.

So instead of thinking of this kind of business improvement purely as an optimization problem based on a series of information transactions, we need to think of it as a problem of power and influence - how can the interests of the students be mobilized to achieve better distribution of scarce resources.

SOA Sourcebook (Open Group)

2009-04-30T11:24:00.003+01:00

I have just got my hands on a copy of the SOA Sourcebook, produced by the SOA Work Group of The Open Group and launched at the Open Group Architecture Practitioner Conference. (See my review of the Open Group Boston Grid.) I also spoke with Chris Harding, Forum Director for SOA and Semantic Interoperability.

Most of the ideas in the SOA Sourcebook have been circulating around the SOA world for some time, and there is (not surprisingly) a lot of overlap (with more or less variation) with material the CBDI Forum has published from 2003 onwards.

Some of this material has also found its way into TOGAF 9, but the SOA working group is not tightly coupled with the Architecture Forum (responsible for TOGAF).

The SOA Sourcebook defines SOA as an architectural style, although it also defines it as a construction technique. As an architectural style, service orientation can apply to the business architecture as well as to the software (application) architecture and the technology (infrastructure) architecture, and you can have any of these without the other two if you want; but as the SOA Sourcebook acknowledges "many people believe that the greatest benefits of SOA are obtained when it is applied to the business architecture".

However, the SOA Sourcebook focuses on the IT component of enterprise architecture - in other words, service orientation as applied to the software (application) architecture - and this is where the view of SOA as a construction technique would also make sense.

The motto of the Open Group is "Boundaryless Information Flow". The SOA Sourcebook shows how SOA can achieve permeable boundaries, but of course that's not the same thing as lack of boundaries. For a deeper analysis of boundaries, you are going to have to look at Chapter 4 of my book on the Component-Based Business. But there are clearly construction issues as well as strategic requirements when joining loosely coupled lumps of enterprise as well as loosely coupled lumps of software.

Inclusion in a long term IT strategy, created by Enterprise Architecture, is the only good justification for large-scale SOA. From an SOA perspective, this might be interpreted as saying that the de facto purpose of EA is to justify SOA. Clearly there is a common interest between EA practitioners and SOA practitioners, which The Open Group will doubtless continue to develop.

Semi-structured processes

2009-04-28T19:36:00.006+01:00

A lot of the focus in the BPM world is on highly repeatable, controlled processes. These processes follow a strict management lifecycle.

analysed and designed by process experts on behalf of process owners,
implemented by process engineers using sophisticated BPM tools and workflow engines
performed by armies of obedient employees or compliant customers
monitored and calibrated by statisticians using sophisticated analytical tools

This is essentially the BPM lifecycle presented by Ann Rosenberg of SAP at the London TOGAF conference today.

In this lifecycle, we can achieve process agility and innovation through rapid redesign and recalibration. If the design and control phases can be make more efficient and effective (for example by being supported by tools), then the set-up costs are reduced, and this improves the economics of scope as well as the economics of scale.

At the other extreme, there may be some processes in an enterprise that are completely adhoc, with no repeatable structure. For example, a decision to merge with a competitor may involve a number of highly complex steps (investigation, evaluation, due diligence and so on) but without any systematic process structure.

Is there something in between these two extremes? Well there certainly should be. In my previous post on Activity-Based Computing, I mentioned four important examples.

Sales. The sales team produces a detailed proposal in response to a complex request from a customer or prospect.

Customer Complaints. Each complaint follows a different path, which depends on the nature and content of the complaint.

Problem solving.

Medical intervention.

All of these processes are semi-structured. The people doing these jobs are given some degrees of freedom to customize the process to their own understanding of the requirements of a particular customer or patient or situation. However, they are still accountable and their performance is measured; and therefore there must be some underlying process structure.

Who is doing the process management? All the participants in the process are participating in the management of the process, and should have access to all the management tools (not just BPM but also BAM and Business Intelligence). Then the job of the process architect is not to lay down the process but to create a collaborative process space in which everyone can be productive and appropriately innovative, with the right balance of variation and standardization, freedom and control.

So the big question for the process architect is not optimizing one process, or even optimizing lots of different processes, but managing the process continuum. In this way, the process architect becomes the enterprise architect.

Update: see Semi-structured processes 2

Decision-Making Services

2009-04-23T12:12:00.004+01:00

James Taylor (ebizQ) explains why we need a separation between process and decision.

Interesting debate on business process and decisions (April 2009)
Decision services (Jan 2007)

One of the reasons for this separation is that decisions are subject to business policies or rules, and such policies are often subject to change over time and/or variation between organizational units. If you code the decisions into process logic, perhaps using BPML to generate process services, then you will have to modify the code or BPLM script whenever the policy changes, and implement different versions for different parts of the organization.

But there are some more flexible ways of architecting decision services.

General-purpose rule engine, driven by a standard policy language.
Clusters of related business decisions (for example planning and scheduling) can be wrapped into a capability service.

The advantage of these approaches is that they help to separate the generic parts of the process from the specific parts, and thus increase reuse and sharing of the generic parts, while increasing accuracy and differentiation in the specific parts.

In his post on business processes and decision services, James discusses some of the technical issues that arise with this separation. It is certainly true that some technical optimization may be desirable in some situations, and this may mean implementing and deploying the process and decision services together as a fairly tightly coupled unit of software (which CBDI SAE calls an Automation Unit). But at the specification level, the logical services should still remain distinct.

For more examples of policy separation, see my articles on Business Modelling for the CBDI Journal. (Some of these require paid subscription.)

Motoring as a Service

2009-04-17T17:53:00.007+01:00

Listening to Power Drive, a BBC radio programme on electric cars. (I heard the live broadcast yesterday evening, and I have now downloaded the podcast from the BBC website to listen again).

I fully expected to hear the voice of Shai Agassi, and I was not disappointed. Until a couple of years ago, Shai was the rising star at SAP: the founder of Netweaver, the champion of service-oriented architecture (SOA) within SAP, frequently talked of as a future CEO. Then he suddenly quit the software industry to work on electric cars.

Shai was one of the first software executives to get the concept of business as a platform of services. (See my post dotBiz from January 2005). He is now talking about motoring as a service, with what he calls a platform-based approach to create a new business model. Shai's company Better Place is building a network infrastructure for rapid charging and battery replacement. (Coverage from about 17 minutes into the BBC programme.)

Shai makes five important points about a service-based approach to motoring.

Establish a separation between car and battery - when you reach a charging spot, you swap your empty battery for a fully charged battery.

Don't solve mileage problem by bigger batteries but by better infrastructure - put charging spots closer together.

We're getting the infrastructure right before we expect people to buy the cars.

Motorists pay for real service (mileage not car)

Shift the industry, not one car at a time. Car-makers should be able to make more profit with electric cars than gasoline-based cars.

The business model is copied from the mobile telephone business. The consumer has a choice between a pay-as-you-go model and a contract model. You can buy a low-mileage contract or an unlimited mileage contract. If you are willing to sign a 24 month contract, you get a better deal for your miles, and you may find a supplier willing to give you a free car.

Of course, this is only economically viable if you can get a large-scale adoption of the new technology. Shai talked about computer simulation models they are using to calculate the business case for a whole country (in terms of reduced oil imports) and to plan the distribution of capacity.

"You're still a software person at heart." says the interviewer and Shai agrees. "At the core of this is a huge software system", he says.

See also Shai Agassi: A bold plan for mass adoption of electric cars (TED Talks)

Towards the Intelligent Business

2009-04-12T16:23:00.007+01:00

What do organizations need to be successful in good times and to survive in bad times?

It's not just about the people (a few talented individuals on large bonuses). And it's not just about the software systems (sophisticated real-time complex event processing or whatever). What matters is how the people and the systems collaborate effectively to address emerging business opportunities and threats, in an agile and innovative manner. I call this Organizational Intelligence.

Organizational intelligence is like human intelligence - it involves the ability to solve problems, to make good judgements, to deal with complex situations in an effective and uncomplicated manner, and to learn from experience.

Many of the technologies I've been talking about on this blog can be used to support Organizational Intelligence, including Business Intelligence (BI), Business Process Management (BPM), Complex Event Processing and Enterprise 2.0.

Conversely, the concept of Organizational Intelligence helps us to work out two things.

To identify the potential contribution of these technologies to business survival (business value).
To determine how these technologies can be combined to support the requirements of a specific organization (architecture).

Technologies for Organizational Intelligence

View more presentations from Richard Veryard.

IT Innovation at Small Bakery

2009-04-09T21:09:00.000+01:00

Can your IT department innovate like this bakery? asks Mark Raskino (Gartner) via Sally Bean. Mark describes an East London bakery that uses Twitter to notify its customers whenever freshly baked bread comes out of the oven.

The system involves a specially designed wall-mounted device that the bakers can operate with one hand, while holding a tray of bread with the other.

Mark clearly doesn't think most IT departments would be comfortable with this kind of system. So I asked Mark and Sally who would lead that kind of innovation in a typical IT department. Would it be the business analysts? The enterprise architects?

The trouble is that many people in those kind of IT roles have been trained to value abstraction and adaptability above everything else, and to avoid thinking about mundane technology when building pure business models. But how often does that kind of abstract thinking produce concrete innovations like this bakery? In many organizations it is quite the opposite - technology-driven opportunistic change to the business model is discouraged because it runs contrary to prevailing thinking. And as Mark points out, corporate IT departments don't like purpose-designed hardware. This system is not designed for adaptability, it is designed to improve the business.

So this project was driven by a marketing agency. Business innovation may depend on the latest technology; but wouldn't it be a disgrace if it turned out that the most innovative companies are those too small to have an IT department?

Scale and Governance

2009-04-09T20:02:00.001+01:00

Following my post Does Britain Need Smaller Banks? Fred Fickling objected to my point that large banks are more difficult to regulate.

"Not sure that the simple answer here is really the right one. Size & corp responsibility aren't necessarily inversely related."

Of course, some large companies can have a highly refined sense of ethics and social responsibility, although this doesn't always protect them from strong disagreements with external lobbyists, as Shell discovered when it wished to decommission the Brent Spar.

In any case, not all large companies can be trusted to regulate their own behaviour in the public interest. Companies pay tax where it suits them, move (or threaten to move) operations from one country to another, shrug off massive fines, and can sometimes behave as if they were above the law, as Fred concedes. In the long term, these legacy corporations may be doomed, but in the short term they can still cause a lot of disruption in the financial ecosystem.

Of course there are no simple answers. I certainly think some companies are too large, and I hope there will always be good opportunities for smaller companies to compete. What I'm calling for is a way of reasoning intelligently about the size of companies, which balances the interests of all stakeholders in a fair and governable manner. I think this is a proper topic for business architecture. And perhaps some of what we've learned from SOA about granularity and governance can be applied to this greater problem domain.

Ford and Fordism

2009-04-08T21:19:00.001+01:00

In his piece Flashlights not Bailouts, Phil Gilbert describes the classic CBD/SOA argument: standardization/rationalization/reuse.

"One of the three (Ford) is in demonstrably better shape than the other two, and it's no mystery why. Two years ago, when he took the reins of Ford, Alan Mulally identified two things that needed to change: parts costs have to go down, and engineering productivity must go up."

"Get it? The white collar workers who design the cars have to move from artisan to engineer, and they need to work together across all the company's platforms to use common parts."

I asked Phil if this was Fordism? Phil responded thus:

"Ford used standardization of task to drive down costs. I'm suggesting 'standardization' of transparency to drive down risk."

Thus Fordism progresses from the economics of scale to the economics of governance. This fits well with the story about SOA I've been telling in this blog. http://tinyurl.com/d3ronn

Does Britain Need Smaller Banks?

2009-04-08T18:20:00.002+01:00

George Osborne, possibly the next Chancellor of the United Kingdom Exchequer, has asked some good questions about the right size of banks. (Robert Peston, Tories to break up banks?, BBC 8th April 2009)

"We cannot allow one part of our economy to behave in a way that puts the rest of the economy at risk when it fails. We need to think deeply about whether we can sustain banks that are not only too big to fail, but potentially too big to bail."

"We should look at whether Britain in fact needs smaller banks. For it would be a bitter irony if we came out of this crisis with a banking system that was even more concentrated and even riskier than the one we had before it."

It is no longer obvious that large banks are any more financially viable than small banks. What seems pretty clear is that large banks are more difficult to regulate. What was the economic logic behind all those mergers and acquisitions in the banking sector anyway? Whatever may have been gained in terms of the economies of scale were totally blown away by the economies of governance.

Umair Haque asks (rhetorically) why bankers never talk about reforming banks. However, as Osborne points out, now that the British government owns a large chunk of the banking sector, there is an opportunity to make radical changes to the sector as a whole, rather than merely controlling the behaviour of each company within the sector. I hope this will allow a serious debate on the scale and granularity of business. See my earlier post on Business Model and Financial Viability.

Update: Here's what Henry Mintzberg has to say about General Motors, in a piece called America's monumental failure of management (Globe and Mail, 16 March 2009).

We hear now about "too big to fail." "Too big to succeed" is more like it. General Motors has been going slowly and painfully bankrupt for decades, managerially as well as financially. The new money will only put off its demise. Americans will have to face this reality sooner or later.

Pay as you drive 3

2009-04-03T04:45:00.003+01:00

Couple of posts from the Smart 421 people on usage-based car insurance.

Their conclusion: it isn't going to save anyone enough money to justify the cost and inconvenience and to overcome the concern for privacy. Adoption only becomes viable when the devices become a commodity (multiple providers, standard interface, routinely fitted to new cars), with a strong carrot/stick from the government (e..g legislation or road charging).

This is pretty close to what I said when the Norwich Union experiment was abandoned last year (Pay as you drive 2).

The Smart 421 bloggers also have some comparisons with Pay-As-You-Go models for other utilities, including domestic energy and water, and telecoms. Telecoms is clearly more complex than energy and water consumption, and has much more complicated pricing schemes. What are the lessons for software pricing - e.g. Anything-as-a-Service?

Enterprise as a Network of Events 2

2009-04-01T21:31:00.005+01:00

Following my post on Enterprise as a Network of Events, Nigel Green drew my attention to something he wrote a couple of years ago on the case for a clear distinction between events and content, in which he points out that "aggregated events become content over time".

As it happens, I touched on something like this in my 1992 book on Information Modelling. At that time, the specific question I was addressing was how to represent the history of organizational change in a data model. There are two basic options.

History is represented as a series of snapshots of the organization at specific times. If you want to know what the change was, you just have to compare two snapshots. (For example, this is how page history is represented on Wikipedia.)
History is represented as a series of changes (differences). Then if you want to know what the state of the organization was at a given time, you just have to aggregate the changes forwards or backwards from a fixed point.

From a purely logical point of view, it doesn't matter which of these two representations you adopt, because you can always derive one from the other. In practical terms, however, you may find that one representation is a lot easier to work with than the other.

What both representations have in common is a definition of information in terms of difference: a difference that makes a difference, as Bateson puts it.

Information as Difference

Hold your hand perfectly still, palm upwards and resting comfortably on a table. With your other hand, drop a small coin into the palm. You will feel the impact, and if the coin is cold, you will feel the coldness of the metal. Soon however, you will feel nothing. The nerve cells don't bother repeating themselves. They will only report to the brain when something changes. Information is difference.

A lizard hunting insects operates on the same principle. The lizard's eye only reports movement to the lizard's brain. If the hunted insect settles on a leaf, the lizard literally cannot see it. But the moment the insect starts to move, whop, the lizard can see it again, and the tongue flickers out and catches it.

But there are differences and differences. Information is difference that makes a difference. You were probably aware, as you dropped the coin into your palm, your eyes told you automatically, without your brain even asking, what the value of the coin was; but you were probably not aware what date it was minted. This is because (unless you are a numismatist) the value of the coin makes a difference to you whereas its date doesn't.

What is it that makes a difference to a lizard, to a numismatist, to you? Surely not the same things. What is information for the lizard is not information for you, and what is information for you is not information for the lizard.

This is why the perspective of information is important. Perspective defines what counts as information at all, perspective defines to whom the information makes a difference.

Enterprise Strategy and Difference

Now what's this got to do with the enterprise? There are two important differentiation questions for any enterprise: how do They differentiate Us, and how do We differentiate Them.

Firstly how do They differentiate Us. In other words what differences in our products and services or organization are (a) going to be noticed by external stakeholders such as customers and (b) going to affect the behaviour and attitudes of these stakeholders.

Sophisticated marketing organizations pay a lot of attention to this kind of thing. Does it make a difference if we put this in a blue envelope? Does it make a difference if we have a female voice here? Sometimes you have to experiment with differences that don't matter, in order to have a refined view of what differences do matter.

Secondly, how do We differentiate Them. What are the strong signals in the environment that we must respond to, and what are the weak signals we might respond to? An organization that responded to every weak signal would be impossibly unstable, so most organizations have operational processes that respond to strong signals, and some form of business intelligence that scans the environment for weak signals and identifies a subset of these signals demanding some kind of response.

Sometimes the appropriate response to some detected difference or discrepancy is some kind of business improvement, such as business process improvement and/or system engineering. Which means that the history of the organization is inscribed into its current structure and processes.

"What's the purpose of this process step?"

"We've done that extra check ever since we got badly burned in the XYZ deal five years ago."

If we knew enough (which of course we don't), we might conceivable infer the current state of the organization from a careful analysis of all the events that have ever happened to it. That would join this post back to where we started. Neat but impractical.

But that's not actually what I'm trying to do. I think understanding an organization as an information-processing system helps us to do two practical and concrete things.

Diagnose the current problems and opportunities facing a complex organization.

Design an event-driven business improvement process, based on business intelligence.

My next question is: what kind of business model supports these challenges.

Mollusc-Driven Architecture

2009-03-25T21:25:00.003Z

Thanks to Brenda Michelson, I have picked up a wonderful example of Smart SOA from Sandy Carter of IBM.

"RFID tagging blue mussels to identify oil leaks -- mussels closes, alert sent, line stopped; no mussels harmed"

According to Steve Núñez of Illation, this is done by Norwegian oil company StatOil. See Benefits of Going Green: Who Would Have Thought? I hope the alert is not sent by snailmail.

For their role in detecting leaks, blue mussels have been dubbed the canaries of the sea. And don't miss the Statoil recipe for Manhatten Mussel Chowder!

Straight-Through Processing 3

2009-03-25T19:09:00.003Z

Where have all the agents gone? asks Seth Godin. He is talking about travel agents, stock brokers, real estate, and so on.

"The problem with being a helpful, efficient but largely anonymous middleman is pretty obvious. Someone can come along who is cheaper, faster and more efficient. And that someone might be the customer aided by a computer."

Straight-through processing - sometimes seen as one of the potential sources of value from service-oriented architecture - is a mixed blessing, as I've pointed out before. There is always a danger of disintermediation, especially if the added value provided by the middleman appears trivial or easy to replicate. As Seth Godin points out, if you are just providing an anonymous human approximation of Google, don't bother - Google does it better.

In some industries, the middleman had become lazy - taking a cut for not doing very much. My local travel agent never gave me any useful information or advice, merely handed out brochures from the travel companies and struggled with the complexities of the booking system; so I stopped using them. And in the financial markets there are still companies that think they deserve a percentage of your pension fund in return for pressing a few buttons now and again.

In the past companies like these have often been able to take advantage of a strategic position in some larger process, extracting "rent" but without creating value. And there are doubtless still many niches in the system ecosystem that can be exploited in this way.

However, service ecosystem niches that allow companies to draw rent without creating value are going to be short-lived, and rightly so. In hard times, the percentage should only go to companies that are providing genuine value. (Actually the same principle applies all the time, but people are more motivated to follow this principle now than they have ever been before.) And ecosystem SOA should help us to make sure of this.

Then business strategy in a dynamic service ecosystem should not be based on finding and maintaining positional advantage, but on creating and maintaining genuinely productive relationships.

Straight-through processing (Feb 2005)
Straight-through processing 2 (April 2005)