Richard Veryard on Computing

Embedding Intelligence in the Business Process - Reprise

2012-04-16T23:11:00.000+01:00

My book on #orgintelligence contains an important chapter on Embedding Intelligence in the Business Process. In terms of software technology there are now at least six aspects to this (and I should be delighted to discover any further aspects).

Embedding business intelligence (BI) into the business process.
Embedding Enterprise 2.0 into the business process.
Embedding knowledge (content) into the business process.
Embedding learning into the business process.
Embedding collaboration into the business process -
Embedding intelligence into business capabilities

There are various hardware and software vendors who offer one or more of these. Here is an unscientific selection.

Intel - Embedded@Intel

Oracle - Fusion CRM Applications - Embedded Intelligence

Textron Defense Systems - "sophisticated embedded intelligence applications that enable warfighters to do more, better and faster, with their current assets". Features of this embedded intelligence apparently include augmented futures, social networking, anthropology, resource optimization, HCI and force multiplication. (Textron website)

Workday - "Embedded Intelligence within business processes allows for real-time contextual insight. Workday enables pre-built or custom worklets to be embedded within selected business processes. These worklets provide relevant, real-time business insight in context at the point of decision." (Press release August 2011)

Machine Analytics, a company in the Boston area, offers a solution methodology called Amitie for embedding intelligence into the business process. Amitie stands for Analyze, Model, IMplement, Test, Interface and Evaluate. The embedding takes place in the final two steps: interfacing the implemented and tested model with the client's business process, and evaluating/monitoring performance of the embedded model within the client's business process environment.

This material is partly based on some posts from November-December 2010.

Software vendors have hidden motives

2011-11-21T23:53:00.000Z

Gartner analyst Dennis Gaughan made some important announcements in Australia recently.

Microsoft, Oracle, IBM, and SAP have hidden motives.
Microsoft mainly wants to protect Windows and Office.
Oracle products don't really work well together.
IBM wants to take over your IT strategy.
SAP confuses customers with pricing.

[Source: Business Insider 21 November 2011]

A cynical view of Gartner would be that such opinions would be suppressed because Microsoft, IBM, Oracle, and SAP are clients of Gartner research. For example, @greenm3 (Dave Ohara) wonders why Gartner would want to spin in this particular way [greenm3]. For my part, I don't think I can recall a major industry analysis firm going on the record so baldly, and all credit to Dennis Gaughan for his frankness.

I'm wondering what these comments imply for the traditional Gartner criteria of "vision" and "ability to execute". How can we separate the commercial and technological capabilities and characteristics of these vendors, and can we always assume that such vendors mobilize what vision and ability-to-execute they may possess in the best interests of their customers? Let's hope Gartner will now pursue this line of analysis further.

Black Swan Blindness

2011-09-01T01:22:00.002+01:00

In my post Black Swans and Complex System Failure, I talked about the architectural implications of some recent disasters, including the Gulf of Mexico oil spillage in 2010 and the partial melt-down in Japanese nuclear reactors following the tsunami in 2011. Both of these disasters involved something that isn't supposed to happen: the simultaneous failure of multiple fail-safe mechanisms.

A new study by Oxford University and McKinsey finds a similar phenomenon in technology investment, where large IT projects may experience spiralling costs as a result of multiple problems occurring simultaneously. According to the researchers, this is up to twenty times more frequent than traditional risk modelling techniques would expect, with one in six large IT projects going over budget by an average of over 200%. Researchers refer to the tendency to disregard rare but high-impact problems/risks as black swan blindness.

As an example, Professor Bent Flyvbjerg cites the collapse of Auto Windscreens, which went into administration in February following a disastrous attempt to implement a new IT system. "Black swans often start as purely software issues. But then several things can happen at the same time - economic downturn, financial difficulties - which compound the risk," he explained.

Professor Flyvbjerg has coined the term Black Swan Management, which currently merits its own Wikipedia page. Simon Moore (author of Strategic Project Portfolio Management) questions whether it is appropriate to use the term "black swan" for something that occurs with a one in six probability, but supports Flyvbjerg's conclusion that when projects go wrong they can go extremely wrong.

Flyvbjerg makes five fairly bland recommendations for avoiding IT project failure, including recruiting a "master builder". Some people may interpret this as an endorsement of the large IT service firms, but these firms have been responsible for some of the most extravagent failures. Is there any evidence that master builders are any more immune from "black swan blindness" than anyone else? Indeed, as a Scandinavian, Flyvbjerg will hardly need reminding of Ibsen's portrayal of madness in the play of the same name.

'Black swans' busting IT budgets (BBC News, 26 August 2011)

Bent Flyvbjerg and Alexander Budzier, Why Your IT Project May Be Riskier than You Think (Harvard Business Review, September 2011, pp. 601-603)

Natasha Lomas, Five ways to stop your IT projects spiralling out of control and overbudget (Silicon.com, 22 August 2011) (pdf)

Brenda Michelson, Complexity, Outliers and the Truth on IT Project Failure (HP Input-Output, 31 Aug 2011)

Simon Moore, Black Swans In Project Management (August 25, 2011)

Ethics of Risk in Public Sector IT

2011-06-08T12:17:00.001+01:00

@tonyrcollins via @glynmoody and @Mark_Antony asks Should winning bidders tell if they suspect a new contract is undeliverable? (8 June 2011) and raises some excellent ethical points about public sector procurement.

One of the functions of good journalism is to hold people and organizations to account. Tony fishes out a speech given in 2004 by Sir Christopher Bland, then chairman of BT, in which he acknowledged incomplete success in previous ventures, and admitted the extraordinary challenges involved in the NPfIT, for which BT had just won three contracts then valued at over £2bn.

There is obviously a difference between something's being extremely difficult and its being impossible. BT executives can fairly claim that they were always open about the chance that it was going to be difficult, and that they didn't know for sure that it was going to be impossible. But at the same time, there is an asymmetry of information here - the supplier is presumably in a better position to assess certain classes of risk than the customer. (Meanwhile, there may be other classes of risk that the customer should know more about than the supplier.)

In my opinion, the ethical issues here are not to do with deliberate concealment of known facts, but of misleading or inadequate assessment of shared risk. The key word in Tony's headline is the word "suspect". So what are the ethics of doubt?

TIBCO platform for organizational intelligence

2011-03-07T00:13:00.002Z

By adding tibbr to its established software portfolio, TIBCO has now extended its range of organizational intelligence technologies. Last week I spoke with Stefan Farestam of TIBCO to discuss the present and future prospects for TIBCO customers linking these technologies together in interesting ways.

We talked about three main technology areas: Complex Event Processing (CEP), Business Process Management (BPM) and Enterprise 2.0. For TIBCO at least, these technologies are at different stages of adoption and maturity. TIBCO's CEP and BPM tools have been around for a while, and there is a fairly decent body of experience using these tools to solve business problems. Although the first wave of deployment typically uses each tool in a relatively isolated fashion, Stefan believes these technologies are slowly coming together, as customers start to combine CEP and BPM together to solve more complex business problems.

Much of the experience with CEP has been in tracking real-time operations. For example, telecommunications companies such as Vodafone can use complex event processing to monitor and control service disruptions. This is a critical business concern for these companies, as service disruptions have a strong influence on customer satisfaction and churn. CEP is also used for autodetecting various kinds of process anomalies, from manufacturing defects to fraud.

One of the interesting things about Business Process Management is that it operates at several different tempi, with different feedback loops.

A modelling and discovery tempo, in which the essential and variable elements of the process are worked out. Oftentimes, full discovery of a complex process involves a degree of trial and error.

An optimization and fine-tuning tempo, using business intelligence and analytics and simulation tools to refine decisions and actions, and improve business outcomes.

An execution tempo, which applies (and possibly customizes) the process to specific cases.

The events detected by CEP can then be passed into the BPM arena, where they are used to trigger various workflows and manual processes. This is one of the ways in which CEP and BPM can be integrated.

Social software and Enterprise 2.0 can also operate at different tempi - from a rapid and goal-directed navigation of the social network within the organization to a free-ranging and unplanned exploration of business opportunities and threats. TIBCO's new product tibbr is organized around topics, allowing and encouraging people to develop and share clusters of ideas and knowledge and experience.

Curiously, the first people inside TIBCO to start using tibbr were the finance people, who used it among other things to help coordinate the flurry of activity at quarter end. (Perhaps it helped that the finance people already shared a common language and a predefined set of topics and concerns.) However, the internal use of tibbr within TIBCO has now spread to most other parts of the organization.

The organization of Enterprise 2.0 around topics appears to provide one possible way of linking with CEP and BPM. A particularly difficult or puzzling event (for example, a recurrent manufacturing problem) can become a topic for open discussion (involving many different kinds of knowledge), leading to a coordinated response. The discussion is then distilled into a resource for solving similar problems in future.

TIBCO talks a great deal about "contextually relevant information", and this provides a common theme across all of these technologies. It helps to think about the different tempi here. In the short term, what counts as "contextually relevant" is preset, enabling critical business processes and automatic controls to be operated efficiently and effectively. In the longer term, we expect a range of feedback loops capable of extending and refining what counts as "contextually relevant".

On the one hand, weak signals can be detected and incorporated into routine business processes. Wide-ranging discussion via Enterprise 2.0 can help identify such weak signals.

On the other hand, statistical analysis of decisions can determine how much of the available information is actually being used. Where a particular item of information appears to have no influence on business decisions, its contextual relevance might need to be reassessed.

The adoption of Enterprise 2.0 within the enterprise raises different challenges to the adoption of CEP and BPM. One reason for this is the tricky question of critical mass. Whereas it is possible to conduct a meaningful pilot for CEP or BPM in a small part of the business, it is much harder to get a sense of how Enterprise 2.0 tools will work across the enterprise from a small pilot, and much harder to see concrete return on investment. However, many of TIBCO's customers already have an objective to implement some form of Enterprise 2.0, and the demand is simply to satisfy this objective in the most effective way.

book now Organizational Intelligence Workshop (14 April 2011)
see also Organizational Intelligence Portal

IT analysis and trust

2011-03-04T11:03:00.003Z

@mkrigsman asks "Trust is the currency that matters most. How many analysts / bloggers deserve it?"

@markhillary replies "surely in the same way as a journalist is trusted, by earning it"

@mkrigsman is particularly concerned about those who write about IT failure. (I'm not sure why he singles out that topic, but I note that the concern arose during a conversation with @benioff, boss of Salesforce.) "When someone writes on IT failures ask "What's their angle?". Usually sensationalism, currying favor, or threatening a vendor." When challenged about his own angle by @njames, @mkrigsman replies "I want to expose *why* projects fail, so we understand magnitude of the problem and can improve."

Trust is clearly a difficult issue for software industry analysts. Unfortunately, Michael's answer to Nigel's challenge cannot prove that he doesn't have a hidden agenda, because the untrustworthy are often just as able as the trustworthy to produce a plausible cover story. If we trust Michael it's not because he can answer the challenge but because of his track record.

We also need to ask - trusted by whom. Software companies might prefer industry analysts to be compliant and predictable, but intelligent software users might regard such analysts as being insufficiently independent. Who would you trust to tell you about Microsoft's new platform - someone who is always pro-Microsoft, someone who is always anti-Microsoft, or someone who has a track record of making both positive and negative comments about Microsoft and its competitors?

Of course, this comment doesn't only apply to industry analysts. Robert Scoble, when he worked for Microsoft, made a point of distancing himself from the party line, and he therefore commanded a different kind of attention and respect than did Bill Gates or Steve Ballmer.

From a simplistic software industry perspective, an analyst who talks about IT success might be regarded as a friend, whereas an analyst who talks about IT failure is potentially an enemy. (This might explain Marc Benioff's wish to challenge the hidden agenda of the latter.) While many software and service companies might adopt the from-failure-to-success rhetoric - "the best way to avoid the risk of failure is to buy our software and hire our consultants" - this is not ideal from a sales perspective.

Mark Hillary appeals to a journalistic ethic, which would presumably include things like balance and transparency. But balance is not always appreciated by those with most at stake. In the past, I have written technology reports on new products, which I regarded as generally positive with a few small caveats. (I don't generally waste my time writing about products that are no good.) But the vendors concerned have often regarded my remarks as highly critical. (Fortunately, this over-sensitivity on the part of software companies is now changing, thanks in part to social media, and companies now understand that a robust debate can be just as beneficial as a highly controlled one-way marketing exercise.)

From a narrow software industry perspective, a trustworthy industry analyst is one who satisfies Simon Cameron's definition of an honest politician - "one who, when he is bought, will stay bought". But from a broader perspective, we should surely prefer to trust those industry analysts with independently critical mind, unafraid to ask awkward questions and publish the answers.

With the large industry analysis firms, the question of trust shifts from personal integrity to corporate integrity. The sales pitch for these firms depends not just on isolated flashes of insight from individual analysts, but on the collaborative intelligence of a community of analysts. Corporate integrity depends not just on transparency about the relationship between the work paid for by software vendors and the independent research consumed by CIOs, but also on a coherent and robust research methodology adopted consistently across the firm, typically supported by an apparatus of surveys and structured questionnaires and checklists and spreadsheets. However, there is a potential disconnect between the routine processing of supposedly objective raw data (this product with this market share in this geography in this time period) and the generation of useful interpretation and opinion, which is where the analytical magic and subjectivity comes in. One example of this magic, Gartner's Magic Quadrant, has been challenged in the courts; Gartner's defence has been that MQ represented opinion rather than fact. (See my post The Magic Sorting Hat is Innocent, Okay?) And the complicated relationship between fact and opinion, and the transparency of reasoning and evidence, is surely relevant to the level of trust that can be invested by different stakeholders in such analyses.

By the way, why am I writing about software industry analysis? Obviously, because I want to expose *why* analysis fails, so we understand magnitude of the problem and can improve. How can software industry analysis deliver greater levels of intelligence and value to the software industry as a whole?

Jeopardy and Risk

2011-02-18T11:13:00.000Z

@Forrester's Andras Cser notes the victory of IBM's Watson computer in a TV quiz game, and asks How Can You Capitalize On This In Risk And Fraud Management?

In his short blogpost, Cser doesn't offer an answer to this question. He merely makes one assertion and one prediction.

Firstly he asserts an easy and superficial connection between the game of Jeopardy and the profession of security, based on "the complexity, amount of unstructured background information, and the real-time need to make decisions." Based on this connection, he makes a bold prediction on behalf of Forrester.

"Forrester predicts that the same levels of Watson's sophistication will appear in pattern recognition in fraud management and data protection. If Watson can answer a Jeopardy riddle in real time, it will certainly be able to find patterns of data loss, clustering security incidents, and events, and find root causes of them. Mitigation and/or removal of those root causes will be easy, compared to identifying them."

As this is presented as a corporate prediction rather than merely a personal opinion, I'm assuming that this has gone through some kind of internal peer review, and is based on an analytical reasoning process supported by detailed discussions with the IBM team responsible for Watson. I'm assuming Forrester has a robust model of decision-making that justifies Cser's confidence that the Jeopardy victory can be easily translated into the fraud management and data protection domain within the current generation of technology. (Note that the prediction refers to what Watson will be able to do, not what some future computer might be able to do.)

For my part, I have not yet had the opportunity to talk with the IBM team and congratulate them on their victory, but there are some important questions to explore. I think one of the most interesting elements of the Watson victory is not the complexity - which other commentators such as Paul Miller of Engadget have downplayed - but the apparent ability to outwit the other competitors. This ability may well be relevant to a more agile and intelligent approach to security, but that's a long way from the simplistic connection identified by Cser. Meanwhile, I look forward to seeing the evidence that Watson is capable of analysing root causes, which would be a lot harder than winning at Jeopardy.

Paul Miller, Watson wins it all, humans still can do some other cool things (Engadget 16 Feb 2011)
IBM's Watson supercomputer crowned Jeopardy king (BBC News 17 Feb 2011)

Technological Identity and Difference

2011-02-01T20:36:00.000Z

Can we trace any link between the Apple Newton and the Apple iPad? When can we regard a technology as "the same again"?

When Apple discontinued its Newton hand-held computer in 1998, one possible explanation for this decision was that it had been "an idea ahead of its time".

Apple's surprise decision to discontinue its Newton hand-held computer ends one of the computer industry's boldest experiments. CEO Steve Jobs' decision halts further development of the Newton technology, which includes the hand-held MessagePad and the portable eMate. An earlier plan to spin off Newton as a separate entity was reversed by Jobs. Many observers interpreted Jobs' move at the time as assigning a role to Newton in Apple's evolving strategy. Apple instead will devote resources to its mainstream Macintosh computer line. Newton has won critical praise for its newer versions, but the product failed to overcome negative publicity on its handwriting recognition flaws. Competitors, such as the PalmPilot device, surpassed Newton in the market. Apple has sold about 200,000 Newtons since 1993, compared to more than one million PalmPilots sold since 1996, according to analysts.

Source: Jim Carlton, Apple drops Newton, an idea ahead of its time (The Wall Street Journal Western Edition, 1998)

It would not be hard to produce one list of the similarities between the Newton and iPad, and another list of the differences. Some people might regard the similarities as more important, and argue that the essential idea was the same. Other people might regard the differences as more important, and argue that the iPad was essentially new, with merely a distant family connection to the Newton.

For many purposes, it doesn't really matter how we talk about this relationship, and some writers may flipflop erratically between identity and difference. But if we wish to understand more deeply how technological ideas develop and evolve and spread, we need a more rigorous basis for talking about technological identity and difference. What are the essential characteristics that we must pay attention to?

Can an idea that was once ahead of its time ever return, or is it always going to be transmogrified by our intervening experience? Can we step into the same technological river twice?

Industry Analysis Transparency

2011-01-24T23:33:00.002Z

@jyarmis asks Should analyst organisations who advise clients on technology purchases be transparent about who pays for their research?

Analysts may trouser cash from vendors for various services - commissioned work, consultancy, membership, sponsorship, webinars, white papers. But we should also be aware of the many non-financial ways in which vendors can influence analysts. Some vendors put a lot of effort into flattering analyst egos, making them think they have privileged access to senior management, giving them exclusive previews, showing enormous respect for their strategic insights; such flattery pays off when the analyst perceives these vendor as brimming with vision and ability to execute. Vendors can also influence analysts by making their job easier, giving them lots of free material that can be reworked into their reports.

See my previous posts on Industry Analyst Coverage and A Value Proposition for Industry Analysis.

Future of IT

2010-11-25T12:30:00.001Z

@klintron summarizes #Forrester #Gartner and #Zapthink on the trends shaping the future of IT. At first sight, these lists appear to offer a random set of interesting but disconnected opportunities (pretexts) for the CIO to get distracted from Business-As-Usual. Is there a way of working these trend fragments into a coherent future vision for Business IT?

Maybe we shouldn't expect a single coherent vision to cover all this stuff, but I believe organizational intelligence provides a useful framework for joining the following fragments, and connecting them to lasting business value.

Collaboration platforms become people-centric (Forrester)
Process-Centric Data and Intelligence (Forrester)
BPM will be Web-2.0-enabled (Forrester)
Business impact of social computing (Gartner)
Complex systems engineering (ZapThink)

Elements

Enterprise Architecture. “Static frameworks give way to continuous business transformation best practices. ” (ZapThink)

Business Process Management. “Organizations begin truly managing their business processes.” (ZapThink)

Social computing. The technologies and principles behind Facebook, Twitter and LinkedIn “will be implemented across and between all organizations.” (Gartner)

Business Pay-Off

“It will unleash yet to be realized productivity growth, it will contribute to economic growth.” (Gartner)

“Organizations ... achieve their goals in the context of an ever-changing business environment.” (ZapThink)

Architectural Framework

Of course, these and similar benefits have been claimed for any number of previous technological innovations. It is not clear from these brief quotes from some of the leading IT analyst firms exactly how they believe specific combinations of specific elements would produce these outcomes, and how a CIO (or software salesperson) could reason about the likely return on investment. I'm guessing they charge buckets for that kind of insight.

Technologies for Organizational Intelligence

View more presentations from Richard Veryard.

Places are still available for my Organizational Intelligence Workshop on December 8th.

Embedding Intelligence into the Business Process 2

2010-11-22T09:52:00.003Z

#orgintelligence #entarch In my previous post, I talked about two aspects of Embedding Intelligence into the Business Process.

Embedding business intelligence (BI) into the business process.
Embedding Enterprise 2.0 into the business process.

In this post, I'm going to talk about two further aspects of this.

Embedding knowledge (content) into the business process.
Embedding learning into the business process.

Embedded Knowledge Content

There are various levels at which knowledge can be embedded in a business process. Some forms of procedural knowledge can be encapsulated as static rules, which can be either written into the process (as software or bureaucratic procedure) or stored in a form that can be easily and automatically referenced by software components or knowledge workers. There is a considerable software literature on so-called business rules - see for example my review of Business Rule Concepts.

More complex forms of knowledge can be represented as models. For example, the business processes associated with operating a complex industrial process or communications network require some representation of the physical structure and processes involved, while business processes in the finance world may use economic models that help to predict market trends and risks. These models may be buried within complicated algorithms, or represented visually in dashboards and control room displays. See my post on OrgIntelligence in the Control Room.

Thirdly there is contextual knowledge - an appreciation of the specific circumstances and general trends relevant to a business decision or action. This kind of knowledge is dynamic and typically requires human mediation and interpretation, although it may be possible to codify and even automate some limited kinds of contextual knowledge. When discussing the contribution of Enterprise 2.0 to the American security services, Dennis Howlett comments that "content without context in process is meaningless". (See my post on Connecting the Dots).

In her post on The Future of HRM Software, Naomi Bloom talks about embedded intelligence that integrates the rule-based and the contextual knowledge into a software agent she calls "Naomi". She claims that embedded intelligence can achieve several things.

It "replaces what we lost" when we reduced or eliminated the interaction between experts and the rest of the organization. (In her piece, the experts are HR professionals.)
It improves upon human embedded intelligence by removing human error.
Automated embedded intelligence improves compliance to rules/policies/regulations and reduces the organization’s exposure to risk.
Commercial Web sites (Landsend.com, Amazon.com) and social Web sites (Wikipedia) set expectations of the embedded intelligence to be found in any self service environment.

In my post Intelligent Knowledge Management, I pointed out the important step from collaborating-in-the-work (for example shared responsibility for decisions) and collaborating-in-the-knowledge (for example, shared responsibility for collecting and interpreting intelligence, connecting the dots). I also advocated a shift of emphasis from knowledge sharing to knowledge embedding - grounding the work in the best available and critically evaluated knowledge, as well as actively seeking well-grounded knowledge to support organizational learning.

One of the ways that enterprise architects can think strategically about business capabilities and business processes is in terms of knowledge intensity - in other words, the quantity and quality of knowledge required in a given capability or process to differentiate the enterprise from its competitors. The "core" activities of an enterprise are those requiring high levels of knowledge intensity and specificity, other activities can be regarded as "peripheral" and may be commoditized or outsourced. See my post on Ecosystem SOA, which draws on the work of Amin and Cohendet.

Embedded Learning

In my post on Learning by Doing, I pointed out that such characteristics as adaptability, agility, flexibility, responsiveness (supposed to be the benefits of various technologies including SOA) imply processes of adaptation and learning. So we need to ask: How do business systems (both organizational and technical) improve? Where is the learning located? What is the nature of the feedback loop?

The learning loop goes through the software developers. (The software development acts as a gate/brake on the learning process.)
A learning process is contained in the software or service. (Learning can take place in real-time, but only for things that have been explicitly anticipated in software development.)
The learning process is distributed across the community usage of the software or service.

In planning for organizational intelligence, we need to think about these learning processes, and how they may be accommodated in any sociotechnical system architecture. Advanced software (from SOA to Enterprise 2.0) gives us new and more flexible ways of implementing such learning processes, but only if we identify the learning requirements properly. We are going to need a business model that includes the learning capabilities as well as the operational capabilities, and an architecture that mobilizes these capabilities in a loosely coupled manner.

Places are still available for my Organizational Intelligence Workshop on December 8th.

Embedding Intelligence into the Business Process

2010-11-19T06:42:00.002Z

Is the business process evolving from bureaucratic workflow towards some form of flexible intelligence? Some of us have been predicting this for a few years now, but there are some hopeful signs that it may finally be starting to happen.

In this post, I'm going to talk about two specific aspects of this.

Embedding business intelligence (BI) into the business process.
Embedding Enterprise 2.0 into the business process.

Embedded BI

The idea of embedded business intelligence has been around for many years. See my blog on Service-Oriented Business Intelligence from September 2005. See also my slideshare presentation.

When software vendors talk about embedded BI, they often mean embedding BI functionality in other pieces of software - for example ERP applications - to allow these applications to produce more interesting reports. There are several niche BI producers in this space, including Jaspersoft, Pentaho and Yellowfin. Brian Gentile of Jaspersoft talks about this in his recent article The BI Revolution: Business Intelligence's Future. TDWI November 10, 2010. For an article explaining the difference between Embedded BI and Integrated BI, see Execution MIH.

For BI to be embedded in the business process, we need to have an understanding of the business process that includes some cognitive task, such as a complex decision, where some business intelligence capability can be used specifically to support this cognitive task. In some cases, the aim might be to make the process faster and more efficient, but more usually the aim is to make the process more powerful and effective.

Embedded BI in this sense can also be related to intelligent event processing, where analytic capability embedded in one process can trigger automatic as well as human responses in other processes. Brian Gentle talks about this in an earlier article The BI Revolution: A New Generation of Analytic Applications. TDWI October 20 2010.

Beyond embedding BI in the business process, we might look forward to a state in which analytics is embedded in the entire enterprise, what Tom Davenport and his colleagues call the Analytic Organization. (See my review of Competing on Analytics.) This is the proper meaning of the term Pervasive BI, which Dave Mittereder defined in 2005 as "empowering everyone in the organization, at all levels, with analytics, alerts and feedback mechanisms" (Pervasive Business Intelligence: Enhancing Key Performance Indicators Information Management Magazine, April 2005).

Embedded Enterprise 2.0

In her piece Time For Enterprise 2.0 To Get Enterprisey, Sandy Kemsley takes a sceptical look at the extent to which Enterprise 2.0 is supporting the core business.

"You hear great stories about social software being used to strengthen weak ties through internal social networking, or fostering social production by using a wiki for project documents, but many less stories about using social software to actually run the essential business processes."

She quotes Andrew McAfee:

[The CIOs] weren’t too worried that their people would use the tools to waste time or goof off. In fact, quite the opposite; they were concerned that the busy knowledge workers within their companies might not have enough time to participate.

And comments:

The fact that the knowledge workers had a choice of whether to participate tells me that the use of social business software is still somewhat discretionary in these companies, that is, it’s not running the core business operations; if it were, there wouldn’t be a question of participation.

It seems to me that there are two possible interpretations of McAfee's remark. Sandy's interpretation is that busy knowledge workers simply don't find time to do any Enterprise 2.0 stuff at all, and she concludes that if the business process can still work without it, then the Enterprise 2.0 stuff is discretionary. An alternative interpretation might be that the business knowledge workers don't have enough time to do enough Enterprise 2.0 stuff to get as much intelligence (requisite variety) into the business process as the business really needs. (I happen to prefer the second interpretation, but I don't know whether this is what McAfee really meant.) In other words, it could be that there a trickle of benefit rather than a decent flow.

I'm presuming that the way Enterprise 2.0 is used within the business process is to support specific cognitive tasks, such as interpreting and making sense of events, and making complex decisions. These tasks may be done by an individual knowledge worker, possibly drawing on knowledge made available by co-workers, or may be done collectively by a network of knowledge workers. The quality of sense-making and decision-making doesn't necessarily increase just because you have more people spending more time on it, but with highly complex business situations the opposite is almost certainly true - the quality will be impaired if you have too few people devoting insufficient time and attention.

But I worry a bit when technology vendors merely invoke the magic words "business process" without demonstrating any real understanding. For example, Sandy's blog links to Klint Finley's piece on Tying Enterprise 2.0 to Business Processes, or Creating New Processes for the Social Enterprise, which doesn't say anything I can recognize as being about business process; as far as I can see, it is largely about activity stream filtering as a technical solution for integrating pieces of software. Finley's piece links in turn to a Monday Musing by R "Ray" Wang which states that

Organizations seeking a marketing edge must digest, interpret, and asses (sic) large volumes of meta data from sources such as Facebook Open Graph.

I think there may possibly be a business process implicit in there somewhere, but exactly how this business process would be supported by Enterprise 2.0 is left to the imagination. I hope "asses" isn't a Freudian slip.

To be clear, I can see how the technologies Klint and "Ray" are talking about might possibly be embedded into a sociotechnical system to support a real business process. But they aren't actually making the connection, nor are they providing any evidence that anyone else is doing so. Even Michael Idinopulos, who at least sounds as if he knows what he is talking about in The End of the Culture 2.0 Crusade? fails to provide any concrete examples. He may have seen some evidence, but he's not telling us. So (not for the first time) it's left to Tom Davenport to say something useful. In a short blogpost for HBR, he provides a couple of examples of what can be done when the social and structuring aspects of technology are combined (Want Value from Social? Add Structure).

Note: some of the larger software vendors have a stake in several of these areas, and are trying to integrate different product lines. For example, IBM adds predictive analytics and social networking in Cognos 10 (SearchBusinessAnalytics.com, 25 Oct 2010). Meanwhile, the niche software providers may be developing interesting partnerships and collaborations - Brian Gentle emails me with a note about the embedding of Jaspersoft within eBuilder, a Swedish provider of an end-to-end B2B suite of Cloud Supply-Chain Management Processes, to produce what they call a Strategic Management Tool.

Places are still available for my Organizational Intelligence Workshop on December 8th.

Collaboration Chasm

2010-11-03T23:55:00.001Z

I've just been looking at a Collaboration Framework published by the CISCO community earlier this year [Insights from the Collaboration Consortium Year One], which uses the well-known Crossing-the-Chasm model of technology adoption, popularized by Geoffrey Moore and loosely based on the work of Everett Rogers. The CISCO term “collaboration chasm” refers to the notion that there is a discontinuity between the use of a limited number of technologies by early adopters and the large-scale adoption by mainstream users.

Rogers himself modelled adoption as a continuous S-curve, but Moore's notion of a chasm is popular with supply-side marketing people, because it suggests a single heroic leap from an experimental product to a commercial success.

In the context of collaboration technologies within the enterprise, the "chasm" metaphor can be interpreted in multiple ways, all of which are discussed or implied in the CISCO document.

The categorical difference between individual early adopters and everyone else. A simplified model of adoption would regard "early adopter" as a personality type, predicting a person's attitude to any kind of innovation, similar to the Adaptor/Innovator scale developed by Michael Kirton. (Rogers himself recognized that a person could easily be an early adopter of one technology and a late adopter of another.) Additionally, some writers may wish to characterize particular organizations as early adopters.

The categorical difference between Generation X and Generation Y. The assumption that because younger people are likely to be more comfortable with certain classes of technology, they will therefore be more positively inclined to the adoption and use of these technologies in the workplace.

The difference between social and workplace use of these technologies. Jagdish Vasishtha thinks this has something to do with personal choice, saying "there is a growing chasm between how people now communicate in their personal space and how they are forced to communicate in a corporate environment" [Crossing the Chasm, May 2009 (pdf)]. The CISCO document points out several reasons why technologies such as social networking or instant messaging don't necessarily transfer easily from one’s personal life to the workplace, and quotes Ray Ozzie on the "chilling effect" of organizational dynamics [Churchill Club, June 2009]. See also Richard Dennison, BT 2.0 Case Study, November 2007.

The step from undirected ("bottom-up") investigation to directed ("top-down") business process change. "Carefully shaping a subset of collaboration initiatives in a top-down fashion to align them with business priorities provides the required structure to scale up an organization’s efforts into the performance stage."

The step from isolated experimental use to integrated use. CISCO describes a 3-stage development strategy (1: Investigative, 2: Performance, 3: Transformational), and positions the "chasm" between stages 1 and 2. For an SAP example of the "chasm" between stand-alone collaboration and embedded collaboration, see Irwin Laazar, Collaboration in Context (May 2010).

"Collaboration creates shifts in the organizational mindset." This might possibly qualify as a second chasm between CISCO stages 2 and 3.

However, there are some misalignments between these different notions. For example, the fact that many young people are familiar with social networking in their private lives doesn't necessarily imply that they will be better able than their parents to use social networking effectively in their professional lives. Effective use in a given social context depends on purpose and style, and social and organizational experience may be more relevant here than technical skill and enthusiasm.

In my work on technology adoption within the enterprise, I make the distinction between broad adoption and deep adoption. Broad adoption basically means that a given product or platform is used by a lot of different people in a lot of places, but this usage may be casual, occasional and uncommitted. Deep adoption means that the product or platform is used intensively, is embedded in processes and working practices, as well as being integrated with other relevant technologies, but may only involve a few people or departments.

The distinction between broad adoption and deep adoption implies two "chasms" at right angles - one between narrow and broad, and one between shallow and deep. The tactics required to encourage broad adoption are significantly different from the tactics needed to implement deep adoption. CISCO's basic 3-step strategy appears to involve crossing both of these chasms at the same time, but the document also refers to some alternative strategies, including a "cultivation" strategy followed by Statoil. Some adoption strategies may permit different aspects of technology adoption to be decoupled; indeed, a number of the examples cited from CISCO's own internal processes involve localized collaboration within specialized processes, although this may be because enterprise-wide cross-process collaboration is harder to explain.

The distinction between broad adoption and deep adoption may also encourage us to look at early adopters more critically. Those who constantly quest for technological novelties may not appreciate or experience the full power of a revolutionary innovation, and may not the best people to lead serious and sustained commitment to its enterprise-wide and system-deep adoption. By the time the organization is moving into CISCO's stage three, the so-called early adopters may have switched their attention and allegiance to something else.

Software Cadence

2010-11-02T10:07:00.003Z

I was intrigued by Bob Muglia's use of the word "cadence" in an interview last week about the future of Silverlight.

“As with anything as it matures, the (delivery) cadence changes.”

Mr Muglia is the Microsoft President in charge of the company’s server and tools business. The interviewer, experienced Microsoft commentator @maryjofoley, interpreted this to mean that the delivery pace of Silverlight is slowing [Mary-Jo Foley, Microsoft: Our strategy with Silverlight has shifted ZDNet 29 October 2010].

Yesterday, Muglia issued a statement in which he apologized for any surprise, controversy or confusion resulting from what he said, and sought to "expand" and "emphasize" some important points. The comments below this statement display a range of responses from reassurance to betrayal - this debate seems to be more about perceived commitment and trust and developer morale than about the technical issues. Muglia didn't repeat the words "maturity" or "cadence", and to date only one comment has appeared demanding that these words be clarified [http://team.silverlight.net/announcement/pdc-and-silverlight/].

Cadence is an ambiguous metaphor in this context. In music, it refers to the last few chords in a piece of music. Traditional cadences were very quick, but Beethoven introduced cadences that continued for a fair while, loudly repeating the last few chords over and over again without developing any new material, and this style is also found in some rock music. An entirely different meaning of the word is found in cycling and running where it means rotation speed - in other words, the number of times the pedals go round, or the number of times your feet strike the ground. I'm guessing Muglia is using the word in this sense, as a metaphor for the frequency of software release.

Just checking Wikipedia (where else?) I learn that elite runners don't vary their cadence much - if they want to go faster or slower, they simply put more or less energy into each step, rather than changing the number of steps - and this might be a relevant pattern for the software industry as well. However, I can certainly think of software products that continue to be upgraded regularly, even though each release seems increasingly trivial, almost empty of meaningful content, as if we were following the Beethoven cadential style.

The relationship between cadence/maturity and developer confidence is a peculiar one. One might think developers would prefer tools and platforms to remain stable, without a constant flurry of new complicating features, but there is clearly a fear that the rest of the developer community will abandon the platform as soon the new features stop coming. In other words, the viability of a platform relies on its not being perceived as mature or even maturing, and therefore a platform that wishes to retain the confidence of its developer community must get progressively more complicated and difficult to use, because of the accumulation of features that have been added for the wrong reasons.

I don't like that conclusion, I hope it's not true, but if it were true it would certainly explain a few things.

ICT for schoolchildren 2

2010-09-22T18:27:00.001+01:00

@glynmoody blogs about Opening up Computer Studies in the UK. The Royal Society is now leading a project to investigate the way computing is taught in school, starting by asking some good questions.

Glyn points out that the Advisory Group is packed with vested interests, including Google, IBM and Microsoft. Glyn would like to see some more Open Source representatives, but of course even the Open Source brigade would still represent the desire to pack schools with more and more kit, without questioning its real educational benefit.

Glyn wants children to be encouraged to explore alternative products (e.g. Microsoft Word, OpenOffice.org, AbiWord) but I can't see that goes very far in helping to understand "the broader concepts that lie behind computing", whatever those might be. What I think would be much more valuable is for children to learn the advantages and disadvantages of writing using a word processor as compared to old-fashioned paper and pencil, searching for information on the Internet as compared to looking in a reference book, and so on. (Just as in maths, they learn how to do sums using a calculator as well as how to do sums not using a calculator. In my day we used slide rules.)

The problem with computing is that, at this level at least, it isn't a real subject. Schoolkids are trained to use the products of, er, Google, IBM and Microsoft, and maybe these are useful lifeskills, but no more than road safety or telephone skills. Even though many of today's school children will end up working in call centres (and according to today's Guardian, a third of call centre employees have a degree), we don't expect them to pass exams in telephone studies. (I probably shouldn't say that - some idiot bureaucrat may read this and think it's a good idea.)

In any case, it is a gross parody of education to imagine that it consists purely of training for working life. Schools don't teach geography in the belief that anyone who is any good at the subject will become an explorer, they teach geography on the basis that everyone needs to know some important stuff about the world and its people, just as everyone needs to know a decent amount about science and history and politics and other people's religions.

Clearly there are some ways in which technology can assist students in learning all kinds of stuff, as well as giving all teachers (not just ICT specialists) some new ways of presenting and explaining their subjects. However, there may be a temptation to use computer-mediated learning not just because it is more fun but also because it is cheaper and more easily standardized, which could bias the school curriculum towards those topics that can be taught most efficiently using the new technologies and so threatens to undermine the overall quality and depth of education.

I hope that the Royal Society study is able to be much more radical than merely optimizing the penetration of technology into the minds of schoolchildren.

See my earlier post ICT for schoolchildren.

From water cooler to Web 2.0

2010-09-16T23:12:00.000+01:00

On a Linked-In group discussion about organizational intelligence (CBDI Forum), Rob Mocking mentioned the water cooler, which prompted some questions and comments from Richard Gilyead and Ian Macdonald. I'm going to post an edited and expanded version of the discussion here.

Rob expressed some scepticism about formal systems for organizational intelligence, and speculated that the water-cooler might be the most important tool for knowledge management. But obviously a literal water cooler can only support a small number of people at a single location. So what is the internet or intranet equivalent, and what are the organizational and cultural requirements for making a metaphorical water cooler work as effectively as a real one?

As Richard asked

Does this virtual model make the "water cooler effect" a myth since the organisation itself may be small but its partners may be dispersed? Is the "water cooler" actually a personal network that spans organisations? What effect does Web 2.0 have on this (like LinkedIn!)?

Let's start by understanding the value of the "water cooler" to the enterprise. The first point is that people don't just rely on formal information systems and dashboards to know what is going on, they also use a range of informal communication mechanisms including casual and serendipitous chit-chat, as well as Management-By-Walking-Around (MBWA). Some of these mechanisms can be replicated or extended by Web 2.0; in any case, the water cooler merely stands in for anywhere (real or virtual) where these exchanges can take place.

Many IT architects concentrate on building and integrating formal systems (although this task is perhaps increasingly delegated to ERP or SaaS vendors and similar) but organizational intelligence raises the question about the relationship between formal systems and informal systems.

But although Web 2.0 may enable all kinds of communication and sharing that weren't possible before, both inside and outside the organization boundaries, I don't see technology as the efficient cause of change, but merely providing support for change in the organization itself and its processes and capabilities.

Richard made an important observation about strong inward-looking implications of the water cooler. Interestingly, the water cooler metaphor echoes a much older idea of the village pump being the locus of social interaction. (Several Bible stories take place near a well.)

Richard also notes that senior executives tend to rely more on traditional personal networks than on Web 2.0. Of course there are some obvious limitations with Web 2.0, at least as currently available. I posted a fictional example of the Old Boy Network on my blog (Social Networking as Reuse) and suggested it might take a while before Linked-In and Facebook can replicate the kind of affordance offered by more traditional methods.

Ian averred that in 30 years of consulting he never came across an organization where people gathered around a water cooler, and asked if it really happened?

The village pump is more likely, assuming an age where time passes more slowly, but sadly grabbing a coffee and taking it back to your desk is more likely. Of course the village pump was also a major transmitter of disease as untreated sewage would have been piled in middens just yards away and polluted the water source - just as the water cooler/coffee machine can be a source for the rapid spread of baseless rumours.

The main problem we seem to have with the traditional methods of networking is that they are not scaleable or interoperable. Each executive has his/her own personal network of friends and information sources, but that typically results in intelligence silos and thus may not be enough in the face of really large and complex problems.

The main problem we seem to have with Internet-based methods is that they are ungrounded. Poor quality information (rumour) has always existed, but now it can be disseminated globally with a single well-timed Tweet. A great deal of Internet discussion lacks rigour, relevance or respect, and is sometimes quite incomprehensible.

The Internet may therefore simultaneously amplify both intelligence and stupidity, is a constant battleground between them. This is now a large part of the environment in which organizational intelligence must operate.

What shape is the Internet?

2010-09-16T13:29:00.000+01:00

These days the "web" metaphor tells us less and less about the true topography of the Internet.

The current discussion started for me when @ironick quoted @Microsoft "The Web is about sites, and your browser should be, too." and asked "What's the difference between a site and an app?" @Cybersal quoted @dtapscott's alternative "web not about sites but platforms 4 collaboration". So @ironick asked "What's the WWW really a web of? sites, apps, pages, data... "

My first observation is that if the Internet is merely a collection of sites, apps, or even platforms, then it's not exactly a web. The word "web" appears to focus our attention on the connections rather than sites themselves. There are of course two kinds of connection that exist in the Internet, which we could roughly categorize as syntactic and semantic. A syntactic link is a hotlink coded in HTML, while a semantic link involves some kind of content relationship. For example, you'll note that in this blogpost I've gone to the trouble to add hotlinks to the tweets by Nick and Sally: if you wanted, you could go directly to Twitter to check their exact words. (Go directly to Twitter, do not pass Google, do not collect 200 cookies).

But even if I hadn't added the hotlinks, you'd still be able to find Nick and Sally and their Tweets, by copying their names or their words into a search engine. So I'm creating a semantic link just by referencing something that exists somewhere on the Internet, even if I don't tell you its exact location.

The original hypermedia experience was largely dependent upon syntactic links. For @ironick "the web still feels like hypermedia 2me: clicking from context (page, song, video, snippet, site, app) 2 cntxt 2 cntxt 2..." I agree that it often still feels like that, but I find that a lot of my Internet browsing these days involves typing terms into search engines, and I don't find myself following long chains of hotlinks. In other words, I tend to regard the semantic links as more interesting and more useful than the syntactic links.

Here are some of the many problems I experience with syntactic links

Sometimes the links aren't provided at all.
Thanks to an aversion to deep linking, many websites only provide a link to the home page.
Sometimes the links don't go direct but via some tedious aggregator or intermediator page. Spurious links whose sole purpose is to manipulate the search engines or generate advertisement traffic.
Sometimes the links take you somewhere boring or irrelevant or obvious (like a Wikipedia page), not-safe-for-work, or someone's idea of a joke (not Rick Astley again).
Often the links are compressed, so you can't see where you are being led. (No, I don't want to watch a YouTube video right now, thank you.)
Often the links contain all sorts of other coded information, to pass contextual information to the receiving website.
And then to cap it all, half the time the links don't work for you anyway, because they are out-of-date, or because the person providing the link has a subscription and you don't, or because there is some kind of context or syntax error.

Of course, there are problems with semantic links as well, above all the danger of over-reliance on the chosen search engine. But I still feel I'm more in control of the experience.

When we talk about the Internet as a world wide web (WWW), the word "web" seems to suggest a network stretching endlessly in all directions, allowing and encouraging the kind of browsing experience Nick mentions. But of course the fly's experience of the spider's web is quite different: being caught in one place, trapped for the benefit of the spider. For a long time, it has been the desire of major internet providers to trap users in one place: this desire is now apparently satisfied whenever users do not stray more than one or two clicks away from their favourite search engine or social networking site. Maybe that's what Microsoft is getting at.

Social Networks and Received Opinion

2010-07-15T17:28:00.000+01:00

@EthanZ spoke about the internet at TED Global in Oxford this week. The video has not been not posted yet, so here are some key quotes from the summary by Jonathan Fildes (BBC News 15 July 2010).

The internet has not become the great leveller that it was once thought it could be. The web is now contrary to the original utopian vision and users focus on information from a handful of wealthy countries. It's making us 'imaginary cosmopolitans'.

Social networks make the problem worse with the majority of people sharing information with folk who share their world-view. Our world-view might actually be narrowing.

Tools like Twitter trap people in so-called "filter bubbles". The internet is too big to understand as a whole, so we get a picture of it that's similar to what our friends see. If you turn to your friends, eventually you get the wisdom of the flock.

The term "filter bubbles" is credited to political activist @elipariser. See Ethan's earlier post Eli Pariser on Filter Bubbles (March 2010).

This phenomenon is important from many perspectives. One question that particularly interests me is the way that these networks can create the illusion of improved intelligence, while actually doing no such thing.

	wish, illusion	actual
information gathering	availability: fast, rich, high quality, unmediated, diverse	homogeneous, filtered
sense-making & decision-making	open, creative	closed, doctrinaire
knowledge	complete, consistent, strong, independent	partial, partisan, weak, received opinion
learning	progressive	pseudo-learning
communication	authentic	vapid

Obviously it would be crazy to write off social networking and the internet as an inevitable producer of these effects - that would be the kind of crude technological determinism that gets the tabloid newspapers bewailing the Perils of Facebook.

Instead, the challenge is both to use the available human and technical networks more wisely, and to develop sociotechnical mechanisms that help to realise the original vision of these technologies and contribute to a greater and better distributed intelligence and understanding. Zuckerman talks ambitiously about mechanisms for amplifying underrepresented voices, and for discovering content through serendipity. He also talks about important new roles - for example curators to collect the content, xenophiles to bridge different cultures, working together to put content into context.

But even if we cannot transform the world overnight, we (ourselves and our organizations) can at least start to use these technologies in a more contingent manner, and with greater awareness of their strengths and weaknesses.

Email newsletters

2010-07-09T02:39:00.000+01:00

"Is email dead?" asks @Graham_Walsh (via @nigelwalsh @leebryant ) noting that Ben & Jerry's are stopping their newsletters, and now using updates on Facebook & Twitter.

Email has always been a poor medium for broadcasting news, for several reasons.

Email is used for delivering many different kinds of content, ranging from important to rubbish. For most people, newsletters (even from companies we are interested in) are considerable less important than direct messages from customers, colleagues and friends, some of which may require an immediate response. Therefore even the most interesting newsletters are likely put aside for later reading.

Many of us receive hundreds of legitimate email newsletters - from an assortment of companies and societies and other organizations we have some vague association with - as well as loads of more spammy stuff. I have set up an automatic filter for the regular ones, which go into a folder called Newsletters, where they may sit for weeks or months before I find time to look at them. Many get deleted unread.

Traditional newsletters contain several items, but this is problematic on email. Most people won't scroll down beyond the first page without good reason. (This was a problem for a company I once worked for, which used to sell advertising space at the top of its newsletters to other commercial organizations, with the result that some readers only ever saw the advertisement and not our own content.)

There is also a design problem. Email clients are generally less sophisticated than browsers, and newsletters that look fine on one computer may be almost unreadable on another. As for reading a traditional newsletter on a mobile phone or handheld device - forget it. (Maybe the iPad?) So people end up producing text versions and HTML versions, and it just gets more complicated without actually solving the problem.

Finally, there is a problem with organizational innovation. Many people who claim to be leading-edge technologists seem mysteriously attached to email as a general-purpose communication mechanism, and reluctant to use the wide variety of alternative mechanisms that might serve a particular purpose more efficiently and effectively - not just when communicating with customers (who might be slow to adopt newer alternatives) but also with their own peers (who have no such excuse). Email becomes a regressive standard for all forms of communication, and there is little willingness to gather evidence about its effectiveness.

There is an expectation that corporate IT will drive innovation in matters technological. But in many large organizations I've worked in, it is the marketing department that is is more likely to drive this kind of initiative. The use of Facebook or Twitter may be a tactical initiative, adopted as an experiment and abandoned if the results are disappointing; but what I'd see as strategic for consumer-oriented marketing is having a flexible communication platform with strong feedback loops to support detailed customer analytics, and that's what I'd expect corporate IT to provide.

Or you could just stick to email, on the grounds that this is safe and familiar to all - business IT alignment interpreted as keeping IT inside the comfort zone of the business.

The Technology is not the Product

2010-07-08T20:21:00.002+01:00

A perennial source of confusion in technology analysis is the false equivalence between technology and technical products. I refer to this as the first asymmetry. (See for example my article with Philip Boxer on Metropolis and SOA Governance.)

This confusion sometimes crops up in adoption studies, where people talk as if the adoption of a technology was the same thing as the adoption of a product, which is clearly not the case. Leading software companies may sometimes employ people to evangelize a generic technology (SOA, Complex Event Processing or whatever) instead of focusing specifically on their own company's own product range - this tactic allows evangelists from rival companies to collaborate in order to expand the credibility (and therefore the market) for the technology as a whole.

We should also note that complex products often contain a bundle of different technical inventions. As I pointed out in my blogpost on the Red Queen Effect, a product may be composed from a large number of components, each of which may be subject to technical innovation. Product innovation is not a simple linear function of technology innovation; a product lifecycle can be extremely short, but most of the underlying technology may be moving much more slowly. Or vice versa.

Thus a new device such as the iPad shouldn't be regarded as a single instance of "new technology" but as a designed product that contains a large number of new and not-so-new technologies. If I buy a device that happens to contain special features for disabled users, based on the latest technology, can I be said to have "adopted" this technology even if I never actually use those features.

Meanwhile, some products fail to realise the potential of the available technology. For example, Plumen, a niche manufacturer of low energy lightbulbs, makes this claim about its competitors, saying "there are CFLs out there that provide a nice colour of light and that turn on quickly, but they are hidden under a sea of cheap, poor quality bulbs that are given away for free" [Plumen website].

Some types of regulation focus on products rather than technologies. See for example Australasian Biotechnology (Notes from November 1998).

Quotes

"The end product is not the technology but the experiences that we make possible." Griffiths on mobile, Brand Republic, March 2005. “One of our philosophies at MobiTV is the experience as the product, not the technology.” Paul Scanlan, interviewed by Connected Planet, November 2008.

"Whatever the product is, it includes the usability of the technology." John Unger Zussman, Docufictionalizing user manuals. InfoWorld, 15 Feb 1982. Available at Google Books.

See also Sameera Banduk, Marketing Manager of Well.ca, explaining why his company received awards in every category except technology.

"We’re not seen as a technical company, because the technology is not the product. The technology is in the background, and it’s mind-blowing, but what you see from the website is the customer experience and the products." [Customer Focus and Geek Cred - An Interview with Well.ca. January 22, 2010]

And finally, see Interview with Sherry Turkle, Frontline, Sept. 22, 2009

Device-Driven Business IT Alignment?

2010-06-18T20:48:00.000+01:00

@LTucci suggests Using the sex appeal of the iPad to push BI reporting in the C-suite (Total CIO, June 2010). @rtolido glosses this as "looking for better business-IT alignment? Get your CEO an iPad".

Linda Tucci talks about "democratizing business intelligence software" and announces that "users can become masters of their own dashboards!" Although giving more power to CEOs is a curious kind of democratization, I can see that allowing CEOs to become masters of their own dashboards could be interpreted as a move toward some kind of business-IT alignment.

But I hope the reference to the iPad is intended to be satirical, because believing that the CEO would be seduced by some device, thus magically achieving business-IT alignment, would not only show fair contempt for the CEO but also trivialize the notion of alignment. This belief appears to be an extreme form of technology fetishism, christened the device paradigm by the philosopher of technology Albert Borgmann.

For a similar kind of satire, see Newsbiscuit's proposal to give the UK Deputy Prime Minister a toy plastic steering wheel.

From Buzz to Actionable Intelligence

2010-05-25T20:00:00.000+01:00

I've been looking at software that tracks and analyses mentions of keywords across the Internet (sometimes called Buzz).

Why would anyone want to do this? The first obvious interest is in tracking mindshare. How many people are talking about your product versus its competitors.

But it's not enough just to count the mentions of your product. When Microsoft launched the Zune, this was almost universally compared with the Apple iPod, so within a day or two there were thousands of webpages mentioning both. But unsurprising information is of little value; what's potentially significant here is not the absolute numbers but the relative shifts.

There are some important questions here about the volatility of buzz data. If mindshare fluctuates, is this a significant movement, or just random noise? The challenge is to build up enough statistical history to be able to set realistic action thresholds, and to identify potentially important weak signals for further investigation.

It might seem useful to know exactly what people were saying about the two products - which one they preferred and why. Until recently it has been almost impossible for software (and not always easy for humans, especially in unfamiliar cultural settings) to distinguish an enthusiastic "brilliant" from a sarcastic "brilliant", but Israeli researchers are now claiming a 77% precision in detecting sarcasm.

Joe McKendrick, New algorithm spots sarcasm in customer testimonials (Smart Planet, May 2010)
MacGregor Campbell, Just what we need: sarcasm software (New Scientist, May 2010)

However, tagging mentions according to sentiment still looks a pretty inexact science. Some vendors operating in this space don't include automated sentiment analysis at all (e.g. ConMetrics ); others provide simple trends only, leaving humans to do the detailed analysis (e.g. Lexalytics).

But never mind the technical detail. The point of this kind of business intelligence is that it is actionable. Companies can get an early indication of the success of a marketing campaign, long before mindshare feeds through into sales.

Because we aren't just interested in product mentions - we can also track discussion of particular design features of the product. How many people are talking about battery life or screen size or capacity or cost? This kind of detailed information helps identify the features that the marketing campaigns should emphasize, and may also feed into product development. Obviously if battery life is the most talked-about feature of this class of product, then that's a valuable item of intelligence for product designers as well as for sales and marketing. (I wonder how easy it would be to integrate this kind of business intelligence with a requirements engineering tool/method such as Quality Function Deployment, or a statistical technique such as MaxDiff? See Eric Almquist and Jason Lee, What Do Customers Really Want?, Harvard Business Review, April 2009)

If you have enough high-quality data, with all the automatic replication and spam stripped out, then you can also track the influence paths across the Internet over time. Not only identifying the pages that talk about the Zune versus iPod, but which pages came out first, and which of the earlier pages are strongly referenced by later pages. Not just individual thought leaders but also communities or geographies - for example, a given buzz might start on university campuses before spreading to other demographic sectors. That tells you where you should conduct market trials if you want rapid dissemination, and also where you should go for a relatively isolated trial of some high-risk venture. It also tells you which websites to watch for potential trouble.

What interests me most about this kind of innovation is not the technical details but the potential for transforming the business process - to develop greater organizational intelligence. Two years ago, Onalytica founder Flemming Madsen laid out a vision in his blog Predicting Sales from Online Buzz (Jan 2008) and Predicting Sales from Online Buzz - 2 (April 2008).

predicting sales, market share and other outcomes
detect changes in competitors’ behaviour
setting targets known as “influence budgets”
using “influence budgets” to predict whether an organization is on track to meet its actual revenue or market share targets, and take remedial action if required

The critical success factor for this kind of business intelligence is when buzz provides a reliable leading indicator for significant business metrics such as revenue and market share, as well as providing a rapid feedback loop for tactical fine-tuning of strategic projects. This depends not only on the technical quality of the service but also how effectively the service is used and integrated into the business process.

But here's the thing I found most exciting. If an organization can develop sufficient confidence in the reliability of the predictions resulting from this kind of business intelligence, then the visible growth of influence and mindshare may enable it to sustain longer-term programmes and campaigns, instead of cancelling projects that don't deliver an immediate commercial return. Some people might imagine that an organization driven by buzz would be excessively short-termist - but the champions of this approach insist that good use of buzz by a truly intelligent organization could have quite the opposite effect.

I have talked to one large organization using this technology, and I'm hoping to publish this as a case study in the near future. In the meantime, I should be delighted to talk to any other organizations, to see what is actually happening in practice.

See also Just Shut Up and Listen, by Kishore S. Swaminathan of Accenture.

From Organizational Stupidity to IT Disaster

2010-05-20T14:23:00.001+01:00

@robbowley and @flowchainsensei have been looking at @tonyrcollins Anatomy of an IT disaster (November 2009).

Tony's article extracted several key points from the Public Accounts Committee report (pdf) on the C-Nomis project. As he points out, C-Nomis is by no means an isolated example of failure, and much the same could be said of other big IT-based change programmes such as the NPfIT. So I thought I'd try and map his key points against the Symptoms of Organizational Stupidity I outlined a few days ago.

On a preliminary analysis of Tony's summary, at least six of these symptoms are strongly indicated, and can be clearly linked to a very poor outcome. I should be very interested to carry out a more detailed analysis.

Denial

Bending the truth. "The programme team running C-NOMIS reported that the programme was delivering on time and to budget, when it was not."
Over-optimistic 'good news' culture.
NOMS significantly underestimated the technical complexity of the project.

Guesswork

No-one was actively monitoring the budget .
NOMS cannot provide the detail.

Meddle

There was no sustained effort by NOMS to simplify and standardise its business processes reflecting management's misplaced confidence in C-NOMIS, their unrealistic expectations of what could be achieved by an IT solution and their underestimation of the time and costs to deliver it.
"Prison and probation information requirements were quite different and each of the 42 probation areas had different ways of working. End-to-end offender management was little more than a concept, and what it meant in practice and the IT needed to support it had not been worked through."

Muddle

Remarkable lack of insight and rigour, coupled with naivety and over-optimism.
No-one has been held to account. ... The vacuum of leadership within NOMS contributed to confusion and created challenges for suppliers and the project team.

Repetition

Poor decision taking and weak project management on many occasions. The same lessons have still not been learnt.
It is deeply depressing that after numerous highly critical PAC reports on IT projects in recent years, the same mistakes have occurred once again.

Short-sighted

Serious failure to understand the magnitude and cost of the changes which would be needed.

Google and Received Opinion

2010-05-20T11:37:00.000+01:00

Brilliant satire from @newsbiscuit : New Google only searches for sites that match your preconceived opinions.

I have long complained that Google provides a systematically distorted way of finding out what is going on, and encourages what A.A. Milne called Thinking with the Majority. This is because Google's page ranking algorithms are basically designed for people who want to ask the same questions as everyone else, and get the same answers. Consequently, Google helps to amplify the circulation of Received Opinion.

The distortion is further amplified by massive duplication of material from a common source. If you search for a topical story, you will often find hundreds of popular websites repeating exactly the same version of events in slightly different words, and unless you are extraordinarily persistent you may never find a website that gets its information from a different source. See my posts on Google and Spin (1, 2).

@roygrubb raises a related concern - that Google remembers and is influenced by our previous searches. This is not just a privacy issue but also a context issue - our interests may switch from one project to the next. For example, let's say I'm working on a project using SAP, so when I'm searching for technical information on this project I may concentrate on material that is relevant to SAP. But I certainly don't want Google to put an implicit SAP filter on my searches, even if some Google engineer thought this would be helpful to me, because that could seriously prejudice my view of the available technology. Worse, this bias might persist (without my knowledge) when I'm working on a completely different assignment.

I can imagine that Google could build some kind of context-awareness into its search algorithms, so it somehow detects when I move to another project. And (to take a more controversial example) if I search for information about some deadly disease, it can try and work out whether I'm suffering from the disease myself (in which case it can sell me health insurance before it's too late) or enquiring on behalf of a friend or client, or whatever. But that's not the point. The point is the increasingly complicated relationship between our tools and our knowledge, which even many technologically literate people seem touchingly naive about.

Are BPM professionals experts in collaboration?

2010-05-17T19:03:00.002+01:00

@ReduxOnline posts a link to a student project How to measure the collaboration of knowledge workers in the enterprise.

1. To understand the factors supporting collaboration between knowledge workers, a questionnaire was sent to BPM professionals. The paper doesn't make clear whether they were being asked about their own personal collaboration, or about their opinions about collaboration in general. In any case, we might imagine that BPM professionals have a particular perspective on collaboration, which might distort the survey.

2. Nearly a quarter of the BPM professionals couldn't make sense of the collaboration model on which the survey was based, and were unable to answer all the questions. Instead of treating this as a sign that there might be a problem with the model, the researchers chose to exclude these from the analysis. They then argue that the remaining responses validate their model.

3. The survey doesn't measure collaboration, it measures opinions about collaboration, from a carefully selected group of knowledge workers. Perhaps not surprisingly, the opinions are pretty consistent with the kind of management literature that these knowledge workers might be expected to have read. Except that the answers about "purpose" were all over the place (which I can well imagine, given the uncertain intentions of the question), so this factor failed a statistical test (Cronbach's Alpha) and could be quietly dropped from the model.

4. The direct relation between collaboration and the performance of an enterprise is not tested, because the questionnaire did not consists of any financial questions. (It would seem that financial information is "sensitive"; collaboration itself presumably isn't.) Let's hope the students manage to extend their research to include questions about the financial situation of an enterprise, allowing them to demonstrate and explain how maturity of collaboration of knowledge workers (as perceived and understood by BPM professionals) might actually help to improve the performance of an enterprise.

5. I generally regard opinion surveys as low-grade research because they usually merely recycle received opinion. While I understand that this may be the easiest and cheapest form of research, especially for students and software industry analysts, I expect to see some acknowledgement of the potential distortion, rather than merely taking the collected opinions at face value.