TotalData

http://mix.chimpfeedr.com/29cb9-TotalData TotalData 2017-08-16T21:38:00+00:00 http://www.chimpfeedr.com/ tag:blogger.com,1999:blog-6106782.post-3772301548727054194 Digital Disruption, Delivery and Differentiation in Fast Food Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2017-08-16T21:38:00+00:00

In the short term, those fast food outlets that offer digital experience and delivery may get some degree of competitive advantage by reaching more customers, with greater convenience. Denny Marie Post, CEO at Red Robin Gourmet Burgers, sees the expansion of third-party delivery services as a strategic priority. So from agility to reach.

But Lenny Comma, CEO of Jack in the Box, argues that this advantage will be short-lived. Longer-term competitive advantage will depend on the quality of the brand. So from assurance to richness.

Stuart Lauchlan, Digital and delivery – which ‘D’ matters most to the fast food industry? Two contrasting views (Diginomica, 16 August 2017)

Related post: Reach, Richness, Agility and Assurance (Aug 2017)

]]> tag:blogger.com,1999:blog-6106782.post-8012294271182461203 Reach, Richness, Agility and Assurance Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2017-08-15T22:11:00+00:00

I first encountered these four dimensions in discussions of net-centricity, which spilled out from the US defence world into the commercial world over ten years ago. Trying to dig up the original material recently, I found a military version in a report written in 2005 by the Association for Enterprise Integration (AFEI) for the Net-Centric Operations Industry Forum (NCOIF).

Going further back, the first two dimensions - reach and richness - had been discussed by Evans and Wurster before the turn of the millennium. They argued that old technologies had forced you to choose (either/or) between reach and richness, whereas the new technologies emerging at that time allowed you to have both/and.

Source: Evans and Wurster 1997

The authors also introduced the concept of affiliation, by which they meant transparency of relationships - for example, knowing whether the intermediary agent is working for you or working for the other side. Or both. And knowing who really wrote all those "customer reviews".

According to the authors, it would be these three factors - reach, richness and affiliation - that would determine the success of e-commerce. Clearly some sectors would be more open to these factors than others - according to The Economist in February 2000, online trade was then dominated by business-to-business (B2B). The three factors identified some of the challenges facing other sectors, including professional services, in going online. As Duncan, Barton and McKellar argued for legal firms, "The Web provides Reach, but offering Richness and the sense of community required for creating and sustaining relationships with visitors could be difficult."

Meanwhile, new architectural thinking had shown ways of resolving the traditional trade-off between speed (agility) and quality (assurance). (A very early version of this was known as Bimodal IT. Some industry analysts are still pushing this idea.)

When agility and assurance were added to reach and richness to produce the four dimensions of net-centricity, affiliation appears to have been divided between community (reach) and trust (assurance). But the importance of affiliation was never entirely forgotten. As Commander Chakraborty observes, "organisational affiliations and culture ... play very significant roles in a networked environment."

So whatever happened to net-centricity? It has been replaced by data-centricity, which, as Dan Risacher argues, is probably a more accurate term anyway. Or as we call it at Reply, TotalData™.

Notes and References

Much of the original material for the NCOW Reference Model is no longer available. This includes the pages referenced from Wikipedia: NCOW (retrieved 8 August 2017). Net-centric concepts were incorporated into DODAF Version 1.5 (April 2007).

Define and Sell (Economist, 24 Feb 2000)

AFEI, Industry Best Practices in Achieving Service Oriented Architecture (SOA) (NCOIF, April 2005)

Devbrat Chakraborty, Net-Centricity to Ne(x)t-Centricity (SP's Navel Forces, Issue 4/2011)

Peter Duncan, Karen Barton and Patricia McKellar, Reach and Rich: the new economics of information and the provision of on-line legal services in the U.K. (16th Bileta Annual Conference, 2001)

Philip Evans and Thomas S. Wurster, Strategy and the New Economics of Information (Harvard Business Review, Sept-Oct 1997)

Philip Evans and Thomas Wurster, Blown to Bits - How the New Economics of Information Transforms Strategy (Boston Consulting Group, 2000) - excerpts. See also reviews by McRae and O'Keefe.

Hamish McRae, The business world: Three factors that lead to successful e-commerce (Independent, 17 November 1999) - review of Evans and Wurster (2000)

Jordan Moskowitz, Richness versus Reach (Service Channel, 29 Jan 2013)

Terry O'Keefe, The strategy of information: Richness and reach (Atlanta Business Journal, 1 November 1999) - review of Evans and Wurster (2000)

Dan Risacher, The Fundamentals of Net-Centricity (a little late) (4 February 2013)

Related Posts: Beyond Bimodal (May 2016), New White Paper - TotalData™ (August 2016)

TotalData™ is a registered trademark of Reply Ltd.]]>

Source: Evans and Wurster 1997

In this post, I want to talk about an interesting example of a flawed data-driven policy. The UK Prime Minister, Theresa May, is determined to reduce the number of international students visiting the UK. This conflicts with the advice she is getting from nearly everyone, including her own ministers.

As @Skapinker explains in the Financial Times, there are a number of mis-steps in this case.

Distorted data collection. Mrs May's policy is supported by raw data indicating the number of students that return to their country of origin. These are estimated measurements, based on daytime and evening surveys taken at UK airports. Therefore students travelling on late-night flights to such countries as China, Nigeria, Hong Kong, Saudi Arabia and Singapore are systematically excluded from the data.

Disputed data definition. Most British people do not regard international students as immigrants. But as May stubbornly repeated to a parliamentary committee in December 2016, she insists on using an international definition of migration, which includes any students that stay for more than 12 months.

Conflating measurement with target. Mrs May told the committee that "the target figures are calculated from the overall migration figures, and students are in the overall migration figures because it is an international definition of migration". But as Yvette Cooper pointed out "The figures are different from the target. ... You choose what to target."

Refusal to correct baseline. Sometimes the easiest way to achieve a goal is to move the goalposts. Some people are quick to use this tactic, while others instinctively resist change. Mrs May is in the latter camp, and appears to regard any adjustment of the baseline as backsliding and morally suspect.

If you work with enterprise data, you may recognize these anti-patterns.

David Runciman, Do your homework (London Review of Books Vol. 39 No. 6, 16 March 2017)

Michael Skapinker, Theresa May’s clampdown on international students is a mystery (Financial Times, 15 March 2017)

International students and the net migration target: Should students be taken out? (Migration Observatory, 25 Jun 2015)

Oral evidence: The Prime Minister (House of Commons HC 833, 20 December 2016)

TotalData™ is a trademark of Reply Ltd. All rights reserved ]]>

Distorted data collection. Mrs May's policy is supported by raw data indicating the number of students that return to their country of origin. These are estimated measurements, based on daytime and evening surveys taken at UK airports. Therefore students travelling on late-night flights to such countries as China, Nigeria, Hong Kong, Saudi Arabia and Singapore are systematically excluded from the data.

Disputed data definition. Most British people do not regard international students as immigrants. But as May stubbornly repeated to a parliamentary committee in December 2016, she insists on using an international definition of migration, which includes any students that stay for more than 12 months.

Conflating measurement with target. Mrs May told the committee that "the target figures are calculated from the overall migration figures, and students are in the overall migration figures because it is an international definition of migration". But as Yvette Cooper pointed out "The figures are different from the target. ... You choose what to target."

Refusal to correct baseline. Sometimes the easiest way to achieve a goal is to move the goalposts. Some people are quick to use this tactic, while others instinctively resist change. Mrs May is in the latter camp, and appears to regard any adjustment of the baseline as backsliding and morally suspect.

If you work with enterprise data, you may recognize these anti-patterns.

In 2008, Chris Martins, then Marketing Director for CEP firm Apama, suggested considering CEP as a prospective "dog whisperer" that can help manage the risk of the technology "dog" biting its master.

But "dog bites master" works in both directions. In the case of Eliot Spitzer, the dog that bit its master was the anti money-laundering software that he had used against others.

And in the case of algorithmic trading, it seems we can no longer be sure who is master - whether black swan events are the inevitable and emergent result of excessive complexity, or whether hostile agents are engaged in a black swan breeding programme. One of the first CEP insiders to raise this concern was John Bates, first as CTO at Apama and subsequently with Software AG. (He now works for a subsidiary of SAP.)

from Dark Pools by Scott Patterson

And in 2015, Bates wrote that "high-speed trading algorithms are an alluring target for cyber thieves".

So if technology is capable of both generating unexpected events and amplifying hostile attacks, are we being naive to imagine we use the same technology to protect ourselves?

Perhaps, but I believe there are some productive lines of development, as I've discussed previously on this blog and elsewhere.

1. Organizational intelligence - not relying either on human intelligence alone or on artificial intelligence alone, but looking for establishing sociotechnical systems that allow people and algorithms to collaborate effectively.

2. Algorithmic biodiversity - maintaining multiple algorithms, developed by different teams using different datasets, in order to detect additional weak signals and generate "second opinions".

John Bates, Algorithmic Terrorism (Apama, 4 August 2010). To Catch an Algo Thief (Huffington Post, 26 Feb 2015)

John Borland, The Technology That Toppled Eliot Spitzer (MIT Technology Review, 19 March 2008) via Adam Shostack, Algorithms for the War on the Unexpected (19 March 2008)

Vamsi Chemitiganti, Why the Insurance Industry Needs to Learn from Banking’s Risk Management Nightmares.. (10 September 2016)

Theo Hildyard, Pillar #6 of Market Surveillance 2.0: Known and unknown threats (Trading Mesh, 2 April 2015)

Neil Johnson et al, Financial black swans driven by ultrafast machine ecology (arXiv:1202.1448 [physics.soc-ph], 7 Feb 2012)

Chris Martins, CEP and Real-Time Risk – “The Dog Whisperer” (Apama, 21 March 2008)

Scott Patterson, Dark Pools - The Rise of A. I. Trading Machines and the Looming Threat to Wall Street (Random House, 2013). See review by David Leinweber, Are Algorithmic Monsters Threatening The Global Financial System? (Forbes, 11 July 2012)

Richard Veryard, Building Organizational Intelligence (LeanPub, 2012)

Related Posts

The Shelf-Life of Algorithms (October 2016)]]>

from Dark Pools by Scott Patterson

The punctuation example is probably a red herring. The analysis of personality will undoubtedly be based on much richer aspects than mere punctuation: Facebook is capable of much more sophisticated analysis, as well as selling data to other organizations for the same purpose.

For example, a Korean study in 2013 found that Facebook activities had predictive power in distinguishing depressed and nondepressed individuals. However, Facebook may not wish to draw too much public attention to such capabilities. (There are some important ethical issues in the use of algorithms to predict mental health issues, for example in recruitment screening, discussed at length by Cathy O'Neil.)

Meanwhile, insurance companies will wish to use any information and insight they can get their hands on, to try and calculate risk more accurately. People may consent to sharing their data if they feel they will benefit personally, or if they are unaware of the possible data uses and implications, but that could just result in discrimination against the people who refuse to share their data. So privacy campaigners may not be reassured by the fact that this particular collaboration has been withdrawn.

Cathy O'Neil, How algorithms rule our working lives (Guardian, 1 Sept 2016)

Sungkyu Park et al, Activities on Facebook Reveal the Depressive State of Users (J Med Internet Res. 2013 Oct; 15(10): e217)

Graham Ruddick, Admiral to price car insurance based on Facebook posts (Guardian, 2 November 2016, 00.01 GMT)

Graham Ruddick, Facebook forces Admiral to pull plan to price car insurance based on posts (Guardian, 2 November 2016, 18.41 GMT)

Related posts
Weapons of Math Destruction (Oct 2016)]]>

mrkwpalmer (TIBCO) invites us to take what he calls a Hyper-Darwinian approach to analytics. He observes that "many algorithms, once discovered, have a remarkably short shelf-life" and argues that one must be as good at "killing off weak or vanquished algorithms" as creating new ones.

As I've pointed out elsewhere (Arguments from Nature, December 2010), the non-survival of the unfit (as implied by his phrase) is not logically equivalent to the survival of the fittest, and Darwinian analogies always need to be taken with a pinch of salt. However, Mark raises an important point about the limitations of algorithms, and the need for constant review and adaptation, to maintain what he calls algorithmic efficacy.

His examples fall into three types. Firstly there are algorithms designed to anticipate and outwit human and social processes, from financial trading to fraud. Clearly these need to be constantly modified, otherwise the humans will learn to outwit the algorithms. And secondly there are algorithms designed to compete with other algorithms. In both cases, these algorithms need to keep ahead of the competition and to avoid themselves becoming predictable. Following an evolutionary analogy, the mutual adaptation of fraud and anti-fraud tactics resembles the co-evolution of predator and prey.

Mark also mentions a third type of algorithm, where the element of competition and the need for constant change is less obvious. His main example of this type is in the area of predictive maintenance, where the algorithm is trying to predict the behaviour of devices and networks that may fail in surprising and often inconvenient ways. It is a common human tendency to imagine that these devices are inhabited by demons -- as if a printer or photocopier deliberately jams or runs out of toner because it somehow knows when one is in a real hurry -- but most of us don't take this idea too seriously.

Where does surprise come from? Bateson suggests that it comes from an interaction between two contrary variables: probability and stability --

"There would be no surprises in a universe governed either by probability alone or by stability alone."

-- and points out that because adaptations in Nature are always based on a finite range of circumstances (data points), Nature can always present new circumstances (data) which undermine these adaptations. He calls this the caprice of Nature.

"This is, in a sense, most unfair. ... But in another sense, or looked at in a wider perspective, this unfairness is the recurrent condition for evolutionary creativity."

The problem with adaptation being based solely on past experience also arises with machine learning, which generally uses a large but finite dataset to perform inductive reasoning, in a way that is non-transparent to the human. This probably works okay for preventative maintenance on relatively simple and isolated devices, but as devices and their interconnections get more complex, we shouldn't be too surprised if algorithms, whether based on human mathematics or machine learning, sometimes get caught out by the caprice of Nature. Or by so-called Black Swans.

This potential unreliability is particularly problematic in two cases. Firstly, when the algorithms are used to make critical decisions affecting human lives - as in justice or recruitment systems. (See for example, Zeynap Tufekci's recent TED talk.) And secondly, when preventative maintenance has safety implications - from aeroengineering to medical implants.

One way of mitigating this risk might be to maintain multiple algorithms, developed by different teams using different datasets, in order to detect additional weak signals and generate "second opinions". And get human experts to look at the cases where the algorithms strongly disagree.

This would suggest that we maybe shouldn't be too hasty to kill off algorithms with poor efficacy, but sometimes keep them in the interests of algorithmic biodiversity. (There - now I'm using the evolutionary metaphor.)

Gregory Bateson, "The New Conceptual Frames for Behavioural Research". Proceedings of the Sixth Annual Psychiatric Institute (Princeton NJ: New Jersey Neuro-Psychiatric Institute, September 17, 1958). Reprinted in G. Bateson, A Sacred Unity: Further Steps to an Ecology of Mind (edited R.E. Donaldson, New York: Harper Collins, 1991) pp 93-110

Mark Palmer, The emerging Darwinian approach to analytics and augmented intelligence (TechCrunch, 4 September 2016)

Zeynap Tufekci, Machine intelligence makes human morals more important (TED Talks, Filmed June 2016)

Related Posts
The Transparency of Algorithms (October 2016)]]>

"There would be no surprises in a universe governed either by probability alone or by stability alone."

"This is, in a sense, most unfair. ... But in another sense, or looked at in a wider perspective, this unfairness is the recurrent condition for evolutionary creativity."

Microsoft uses the data we collect to provide you the products we offer, which includes using data to improve and personalize your experiences. We also may use the data to communicate with you, for example, informing you about your account, security updates and product information. And we use data to help show more relevant ads, whether in our own products like MSN and Bing, or in products offered by third parties. (retrieved 25 October 2016)

Facial recognition software is big business, and high quality image data is clearly a valuable asset.

But why would 85 million people go along with this? I guess they thought they were just playing a game, and didn't think of it in terms of donating their personal data to Microsoft. The bait was to persuade people to find out how old the software thought they were.

The Daily Mail persuaded a number of female celebrities to test the software, and printed the results in today's paper.

Computer"tell yr age" programme on my face puts me 69 https://t.co/EhEog5LQcN Haha!But why are those judged younger than they are so pleased
— mary beard (@wmarybeard) October 25, 2016

Talking of beards ...

. @futureidentity If we ever reach peak data, advertisers will check photos before advertising beard accessories #personalization #TotalData
— Richard Veryard (@richardveryard) April 1, 2016

. @futureidentity So, did you ever buy that right-handed beard brush? #PeakHipster #Sinister https://t.co/kESqmUooNk #CISNOLA cc @mfratto
— Richard Veryard (@richardveryard) June 8, 2016

Kyle Chayka, Face-recognition software: Is this the end of anonymity for all of us? (Independent, 23 April 2014)

Chris Frey, Revealed: how facial recognition has invaded shops – and your privacy (Guardian, 3 March 2016)

Rebecca Ley, Would YOU dare ask a computer how old you look? Eight brave women try out the terrifyingly simple new internet craze (Daily Mail, 25 October 2016)

Related Post: Another 20 million faces (January 2018)

TotalData™ is a trademark of Reply Ltd. All rights reserved

]]>

Microsoft uses the data we collect to provide you the products we offer, which includes using data to improve and personalize your experiences. We also may use the data to communicate with you, for example, informing you about your account, security updates and product information. And we use data to help show more relevant ads, whether in our own products like MSN and Bing, or in products offered by third parties. (retrieved 25 October 2016)

Computer"tell yr age" programme on my face puts me 69 https://t.co/EhEog5LQcN Haha!But why are those judged younger than they are so pleased
— mary beard (@wmarybeard) October 25, 2016

Talking of beards ...

. @futureidentity If we ever reach peak data, advertisers will check photos before advertising beard accessories #personalization #TotalData
— Richard Veryard (@richardveryard) April 1, 2016

. @futureidentity So, did you ever buy that right-handed beard brush? #PeakHipster #Sinister https://t.co/kESqmUooNk #CISNOLA cc @mfratto
— Richard Veryard (@richardveryard) June 8, 2016

]]> tag:blogger.com,1999:blog-6106782.post-7794810704259587309 Why does my bank need more personal data? Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2016-08-07T13:33:00+00:00

At the end, the cashier asked me for my nationality. British, as it happens. Why do you want to know? The cashier explained that this enabled a security control: if I ever bring my passport into a branch as a form of identification, the system can check that my passport matches my declared nationality.

Really? Really? If this is really a security measure, it's a pretty feeble one. Does my bank imagine I'm going to say I'm British and then produce a North Korean passport? Like a James Bond film?

After she had explained how the bank would use my nationality data, she then asked for my National Insurance number. I declined, choosing not to quiz her any further, and left the branch planning to write a stiff letter to the head of data protection at the bank's head office.

As a data expert, I am always a little suspicious of corporate motives for data collection. So the thought did occur to me that my bank might be planning to use my personal data for some purpose other than that stated.

Of course, my bank is perfectly entitled to collect data for marketing purposes, with my consent. But in this case, I was explicitly told that the data were being collected for a very narrowly defined security purpose.

So there are two possibilities. Either my bank doesn't understand security, or it doesn't understand data protection. (Of course there will be individuals who understand these things, but the bank as an organization appears to have failed to embed this understanding into its systems and working practices.) I shall be happy to provide advice and guidance on these topics.

]]>

It outlines four dimensions of TotalData™ - reach, richness, assurance and agility - and presents a Value Chain from Raw Data to the Data-Fueled Business.

TotalData™: Start making better use of Data (html) (pdf)

(Now I need to write some more detailed stuff, based on a few client projects.)

TotalData™ is a trademark of Reply Ltd. All rights reserved ]]> tag:blogger.com,1999:blog-7415430.post-3548667937394415722 From Networked BI to Collaborative BI Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2016-04-03T08:30:00+00:00

The Cloud BI and analytics vendor Birst has now produced a similar matrix to explain what it is calling Networked BI, placing it in the top right quadrant. Gartner has been talking about Mode 1 (conventional) and Mode 2 (self-service) approaches to BI, so Birst is calling this Mode 3.

While there are some important technological advances and enablers in the Mode 3 quadrant, I also see it as a move towards Collaborative BI, which is about the collective ability of the organization to design experiments, to generate analytical insight, to interpret results, and to mobilize action and improvement. This means not only sharing the data, but also sharing the insight and the actioning of the insight. Thus we are not only driving data and analytics to the edge of the organization, but also developing the collective intelligence of the organization to use data and analytics in an agile yet joined-up way.

I first mentioned Collaborative BI on my blog during 2005, and discussed it further in my article for the CBDI Journal in October 2005. The concept started to gather momentum a few years later, thanks to Gartner, which predicted the development of collaborative decision-making in 2009, as well as some interesting work by Wayne Eckerson. Also around this time, there were some promising developments by a few BI vendors, including arcplan and TIBCO. But internet searches for the concept are dominated by material between 2009 and 2012, and things seem to have gone quiet recently.

Previous posts in this series

Service-Oriented Business Intelligence (September 2005)
From Business Intelligence to Organizational Intelligence (May 2009)
TIBCO Platform for Organizational Intelligence (March 2011)

Other sources

Gartner Reveals Five Business Intelligence Predictions for 2009 and Beyond (Gartner, January 2009). Dave Linthicum, Let's See How Gartner is Doing (ebizQ, May 2009)

Chris Middleton, Business Intelligence: Collaborative Decision-Making (Computer Weekly, July 2009)

Ian Bertram, Collaborative Decision-Making Platforms (Gartner 2011)

Wayne Eckerson, Collaborative Business Intelligence: Optimizing the Process of Making Decisions (April 2012)

Monique Morgan, Collaborative BI: Today and Tomorrow (arcplan, April 2012)
Tiemo Winterkamp, Top 5 Collaborative BI Solution Criteria (arcplan, April 2012)

Cliff Saran, Prepare for two modes of business intelligence, says Gartner (Computer Weekly, March 2015)

The Future of BI is Networked (Birst, March 2016)

Updated 21 April 2016 (image corrected) ]]>

UNICOMSeminars Data Analytics conference next week. Exploring the Business Value of Predictive and Real-Time Analytics (London, 2 December 2015)

A lot of the obvious applications of real-time analytics are in fraud detection and predictive maintenance. I shall be talking about some of the things I’ve been doing recently in the retail and consumer sector, using rich consumer data to drive real-time personalized engagement with the consumer across multiple touchpoints. We have been exploring ways to combine real-time analysis of the consumer’s current state (e.g. current location, what products they are currently looking at, readiness to buy, etc.) with a rich understanding of what one might call the consumer’s “purchasing genes” – for example, do they like to spend a long time reviewing alternative products before purchasing, do they like to wait for a special offer or voucher before buying, or on the other hand do they like to be the first in their social network to have a given product. This is a lot more complex than simply putting them into a fixed number of “consumer segments”.

Based on this analysis, it is possible to select an appropriate “next action” – for example, selecting the appropriate banner to display to the consumer when visiting the website, or the right topic of conversation for a human customer services agent.

Thus predictive analytics are helping retail as it moves from omnichannel commerce (which joins up the buying transaction between the online and the physical world) to omnichannel engagement (which joins up all aspect of the relationship with the consumer).

Omnichannel Commerce
(Systems of Record)

joins up the buying transaction between the online and the physical world

Omnichannel Marketing
(Systems of Engagement)

joins up all aspects of the relationship with the consumer

Given the large volumes of data involved, and the reliance on legacy systems to produce and process the data, we are not yet seeing this analysis being completely done in real-time. However, there are some critical factors that have to be done in real-time. For example, as soon as the consumer buys something, our clients want to stop trying to sell it, and move to a post-sales scenario. (In comparison, even the great Google is still showing me advertisements based on what I was browsing three weeks ago. Fail!)

Over the next couple of years, as the technology gets better, the data scientists get even smarter, and the marketing people get more sophisticated, we may expect an increasing proportion of the analysis to be done in real-time, using machine learning as well as more sophisticated analytics tools.]]>

Omnichannel Commerce
(Systems of Record)

joins up the buying transaction between the online and the physical world

Omnichannel Marketing
(Systems of Engagement)

joins up all aspects of the relationship with the consumer

#WearableTech #InternetOfThings Once upon a time, the wires in an undergarment merely provided structural support. Now, people may have all sorts of wires and wireless devices hidden under their clothing. Here are some interesting examples.

The Foxleaf Bra delivers cancer-fighting drugs through the wearer's skin.
The @tweetingbra reminds women to examine themselves. (?)
The Lumo Lift helps improve posture through app-enabled coaching.
Various manufacturers (including Clothing+, OMsignal and SmartLife) produce health vests and sportswear packed with monitors to track your heart rate, breathing rate and the amount of calories you've burnt.

We are now encouraged to account for everything we do: footsteps, heartbeats, posture. Until recently this kind of micro-attention to oneself was regarded as slightly obsessional, nowadays it seems to be perfectly normal. And of course these data are collected, and sent to the cloud, and turned into someone else's big data. (Good luck with those privacy settings, by the way.)

If a device is classed as a medical device, it will be subject to various forms of accreditation and regulation. For this reason, many device makers will be careful to avoid any specific medical claims, but devices that offer some health advice are considered a borderline area.

Another borderline area is hi-tech underpants that protect men from the evil rays allegedly produced by all those wireless devices. Especially the radiation from mobile phones. (Including the Bluetooth that links your underwear to your smartphone.) One brand of underpants that claims to use a mesh of pure silver to create a Faraday cage around the genitals has been banned by the UK Advertising Standards Authority from making any medical claims.

Or maybe you could just switch the whole lot off.

The Wearable Medical Device in Your Future…Is Now! (Marketing Research Association, 28 April 2015)

Jennie Agg, The hi-tech bra that helps you beat breast cancer - and other clothes that can treat or prevent illness (Daily Mail, 10 March 2015)

Sarah Blackman, Student designs cancer-fighting bra (Lingerie Insight, 10 Feb 2015)

Britta O'Boyle, SmartLife clothing claims to make sure you never miss a beat (Pocket-Lint, 12 March 2015)

Rob Crilly, Hi-tech pants "protect sperm from phone waves" (Telegraph 22 October 2014)

Julie Papanek, How Wearable Startups Can Win Big In The Medical Industry (TechCrunch, 19 Feb 2015)

Hannah Jane Parkinson, Lumo Lift review: posture-tracking gadget is a straight shooter (Guardian, 14 November 2014)

Helen Popkin, Tweeting bra exposed: Genuine support or publicity lift? (NBC News 25 October 2013)

Meera Senthilingam, How a high-tech bra could be your next doctor (CNN, 11 May 2015)

Brendan Seibel, High-Tech Underwear for Adventurous Geeks (21 April 2010)

Mark Sweney, Hi-tech underwear advert banned (Guardian 13 August 2014)

Dan Sung, World Cancer Day - The Real Wonderbra (Wearable, 14 Feb 2015)

Related Posts Have you got big data in your underwear (December 2014)]]>

The Foxleaf Bra delivers cancer-fighting drugs through the wearer's skin.
The @tweetingbra reminds women to examine themselves. (?)
The Lumo Lift helps improve posture through app-enabled coaching.
Various manufacturers (including Clothing+, OMsignal and SmartLife) produce health vests and sportswear packed with monitors to track your heart rate, breathing rate and the amount of calories you've burnt.

The Wearable Medical Device in Your Future…Is Now! (Marketing Research Association, 28 April 2015)

@tetradian reckoned this is a great example of #bizmodel #bigdata for mass-uniqueness. But I didn't see this example the same way: I don't see anything here that turns Mass Customization into what Tom likes to call Mass Uniqueness.

Tom's favourite example of "mass uniqueness" is Picasso. I bet the algorithm couldn't find a bra for the breasts of Picasso's Demoiselles (NSFW).

Breasts of Picasso’s Demoiselle (NSFW)…

A single questionnaire, even from a million women, doesn't get into the big data league. Maybe it would when they start analysing pictures and videos of customer breasts, rather than relying on a simple questionnaire.

Or if the company were to fit sensors to its underwear, monitoring stretch during a range of activities, collecting millions of data points every minute via the Internet of Things.

Do you think I'm joking? Microsoft is working on a Smart Bra, which will monitor the mood of the wearer and detect stress. The Daily Mail suggests that this will help women to lose weight.

"To stop women reaching for the cookie jar when things hit a low, Microsoft's new prototype bra predicts when the wearer is likely to comfort eat and warns against it. The software company's high-tech undergarment features sensors in the cup pockets and side panels that detect changes in heart rate, skin temperature and stress levels - apparent precursors to overeating. All of the data is then streamed via Bluetooth to a smartphone app providing real-time 'mood-triggered eating' alerts."

Now that's what I call big data. Scary, huh?

Jillian Goodman, Cup Size Isn’t Everything (Fast Company, October 2014)

Tom Graves, On Mass Uniqueness (23 May 2014)

April Joyner, Big Data: Coming Soon to Your Bra? (Fast Forward, 6 September 2014)

Hayley Krischer, The underappreciated artistry of the professional bra fitter (Guardian 4 June 2015)

Sadie Whitelocks, Supporting your body in more ways than one! The high-tech bra designed to stop women from comfort eating (Daily Mail, 28 November 2013)

Microsoft working on a smart bra to measure mood (BBC News, 3 December 2013)

See also Towards the Internet of Underthings (November 2015), Weaving in three dimensions (November 2015)

Updated 15 November 2015]]>

Breasts of Picasso’s Demoiselle (NSFW)…

"To stop women reaching for the cookie jar when things hit a low, Microsoft's new prototype bra predicts when the wearer is likely to comfort eat and warns against it. The software company's high-tech undergarment features sensors in the cup pockets and side panels that detect changes in heart rate, skin temperature and stress levels - apparent precursors to overeating. All of the data is then streamed via Bluetooth to a smartphone app providing real-time 'mood-triggered eating' alerts."

Now that's what I call big data. Scary, huh?

IBA Group.

Peter cites an Ovum survey showing that Customer Satisfaction is now the number one concern of management, and argues for what Ovum calls Intelligent CRM. (CA announced something under this label back in October 2000. Other products are available.)

Mark says that CRM and Big Data are widely misunderstood, which is certainly true. My own opinion is the first misunderstanding is to think CRM is about managing THE relationship with THE customer, and I completely agree with Clayton Christensen (via Sloan) that this isn't enough. What we really need to focus on is the job the customers are trying to get done when they use your product or service.

Who is good at CRM? Peter cites an example of a professor of marketing who got a personalized service at a certain chain of hotels and has been talking about it ever since. (That's a pretty good coup for the hotel, if we take the story at face value.) Mark cites the video game market, where both the console manufacturers and the large game publishers are able to collect and analyse huge quantities of consumer behaviour.

Is CRM with Big Data merely a new way of taking advantage of customers? Although most people seem oblivious to the privacy and trust risks, the Wall Street Journal this week suggested that the consumer is becoming more savvy and less susceptible to exploitative loyalty schemes and promotions. This might help to explain why Tesco, once a master of the science of retail, now seems to be faltering.

If there is a sustainable business model based on CRM and Big Data, it must surely involve using these technologies to engage intelligently, authentically and ethically with customers, rather than imagining that these technologies can provide a quick fix for stupid organizations to take advantage of compliant customers.

Related Blogs

Customer Orientation (May 2009)

The Science of Retail (April 2012)

Other Articles

Martha Mangelsdorf, Understanding your customer isn't enough (Sloan Review May 2009)

Shelly Banjo and Sara Germano, The End of the Impulse Shopper (Wall Street Journal 25 November 2014)

Intelligent CRM

AI-CRM "An intelligent CRM system with atuo-learning-tunning engine (sic), Aichain offers the most widely used open source business intelligence software in the world." Last updated March 2013

CA rolling out customer relationship management software (ComputerWorld October 2000)

IBA Group "maintains its focus on IT outsourcing that has become a strategy for many organizations seeking to improve their business processes"

]]>

orgintelligence The recent appointment of an algorithm to a Board of Directors raises the spectre of science fiction becoming fact. Although many commentators regarded the appointment as a publicity stunt, there has always been an undercurrent of fear about machine intelligence. Even the BBC (following Betteridge's Law of Headlines) succumbed to the alarmist headline Could a big data-crunching machine be your boss one day?

There are several useful ways that an algorithm might contribute to the collective intelligence of a Board of Directors. One is to provide an automated judgement on some topic, which can be put into the pot together with a number of human judgements. This is what seems to be planned by the company Deep Knowledge Ventures, whose Board of Directors is faced with a series of important investment decisions. Although each decision is unique, there are some basic similarities in the decision process that may be amenable to automation and machine learning.

Another possible contribution is to evaluate other board members. According to the BBC article, IBM Watson could be programmed to analyse the contributions made by each board member for usefulness and accuracy. There are several ways such a feedback loop could enhance the collective intelligence of the Board.

Retrain individuals to improve their contributions in specific contexts.
Identify and eliminate individuals whose contribution is weak.
Identify and eliminate individuals whose contribution is similar to other members. In other words, promote greater diversity.
Enable trial membership of individuals from a wider range of backgrounds, to see whether they can make a valuable contribution.

Organizational Intelligence is about an effective combination of human/social intelligence and machine intelligence. Remember this when people try to develop an either-us-or-them narrative.

#QTWTAIN

Jamie Bartlett, Will Artificial Intelligence put my job at risk? (Spectator 6 June 2014)

Adrian Chen, Can an Algorithm Solve Twitter’s Credibility Problem? (New Yorker 5 May 2014)

John Rentoul, Will Artificial Intelligence put my job at risk? (Independent 6 June 2014)

Richard Veryard, Does Cameron's Dashboard App Improve the OrgIntelligence of Government? (23 January 2013)

Matthew Wall, Could a big data-crunching machine be your boss one day? (BBC News 9 October 2014)

Retrain individuals to improve their contributions in specific contexts.
Identify and eliminate individuals whose contribution is weak.
Identify and eliminate individuals whose contribution is similar to other members. In other words, promote greater diversity.
Enable trial membership of individuals from a wider range of backgrounds, to see whether they can make a valuable contribution.

Organizational Intelligence is about an effective combination of human/social intelligence and machine intelligence. Remember this when people try to develop an either-us-or-them narrative.

Other Sources

Algorithm appointed board director (BBC News 16 May 2014)

Bud Caddell, Your boss might be better as an algorithm (Quartz, 16 November 2017)

Related Posts

Intelligence and Governance (Feb 2013)

Link added 11 December 2017]]> tag:blogger.com,1999:blog-7415430.post-8865774453325207109 Does Big Data Release Information Energy? Richard Veryard https://plus.google.com/107891026040206160712 noreply@blogger.com 2014-04-26T10:13:00+00:00

michael_saylor of #MicroStrategy says that the Information Revolution is about harnessing "information energy" (The Mobile Wave, p 221). He describes information as a kind of fuel that generates "decision motion", driving people - and machines - to make a decision and take a course of action.

We already know that putting twice as much fuel into a vehicle doesn't make it twice as fast or twice as reliable. (Indeed, aeroplanes sometimes dump fuel to enable a safer landing.) But Saylor explains that information energy is not the same as physical energy.

1. Information energy doesn't follow conservation laws. Information can be created, consumed repeatedly, but never depleted or destroyed. (Unless it is lost or forgotten.)

2. Whereas physical energy is additive, the energy content of information is exponential.

3. The value of information depends on its use, and who is using it.

Let's look at his example.

"Total wheat production for a single year is valuable information; but total wheat production for ten years, combined ten years of rainfall data and ten years of fertilizer represents thirty times more data droplets, but probably contains one hundred times more information energy, because it shows trends and correlations that will drive a greater number of decisions." (pp 221-2).

In other words, thirty times as much data produces a hundred times more information. He doesn't say this extra information MAY drive more decisions, he says it WILL drive more decisions. In other words, the Information Revolution (and our increasing reliance on tools such as MicroStrategy's products) is a historical inevitability.

But is it really true that more data produces more information in this exponential way? In practice, there is a depreciation effect for historical or remote data, because an accumulation of small changes in working practices and technologies can make direct comparison misleading or impossible. So even if the farmer had twenty years' worth of data, or shared data from thousands of other farmers, it would not necessarily help her to make better decisions. Five years' data might be almost as good as ten years'.

Data is moving faster than ever before; we're also storing and processing more and more of it. But that doesn't mean we're just hoarding data, says Duncan Ross, director of data sciences at Teradata, "The pace of change of markets generally is so rapid that it doesn't make sense to retain information for more than a few years." (Charles Arthur, Tech giants may be huge, but nothing matches big data, Guardian 23 August 2013)

According to Saylor, the key to releasing information energy is mobile technology.

"The shocking thing about information is not how much there is, but how inaccessible it is despite the immense value it represents. ... Mobile computing puts information energy in hands of individuals during all waking hours and everywhere they are." (p 224)

What kind of decisions does Saylor imagine the farmer needs to make while sitting on a tractor or milking the cows? Obvious it would be useful to get an early warning of some emerging problem - for example an outbreak of disease further down the valley, or possible contamination of a batch of feed or fertilizer at the factory. But complex information needs interpretation, and most decisions require serious reflection, not instant reaction.

So it is not clear that providing instant access to large quantities of information is going to improve the quality of decision-making. And giving people twice as much information often leads to further procrastination. Surely the challenge for MicroStrategy is to help people deal with information overload, not just add to it?

Furthermore, as I said in my post Tablets and Hyperactivity (Feb 2013), being "always on" means that you never have long enough to think through something difficult before you are interrupted by another event. There is always another email to attend to, there is always something happening on Twitter or Facebook, and mobile devices encourage and reinforce this kind of hyperactivity.

Saylor concludes that "the acid of technology etches away the unnecessary" (p 237). If only this were true.

Related posts

Service-Oriented Business Intelligence (September 2005)
On The True Nature of Knowledge (April 2014)

Updated 19 June 2014 ]]>

"Total wheat production for a single year is valuable information; but total wheat production for ten years, combined ten years of rainfall data and ten years of fertilizer represents thirty times more data droplets, but probably contains one hundred times more information energy, because it shows trends and correlations that will drive a greater number of decisions." (pp 221-2).

"The shocking thing about information is not how much there is, but how inaccessible it is despite the immense value it represents. ... Mobile computing puts information energy in hands of individuals during all waking hours and everywhere they are." (p 224)

Related posts

Service-Oriented Business Intelligence (September 2005)
On The True Nature of Knowledge (April 2014)

Updated 19 June 2014 ]]> tag:blogger.com,1999:blog-7415430.post-9197334965563726660 Predictive Analytics for the Smart Consumer? Richard Veryard https://plus.google.com/107891026040206160712 noreply@blogger.com 2014-04-23T23:42:00+00:00

#CW500 If merchants can use predictive analytics to get more out of the customer, why can't the customer use predictive analytics to get more out of the merchant?

In December 2012, I reported on a subscription-based service from decide.com, which predicted future retail price changes (based on retailers' past behaviour) and encouraged its members to use these predictions to optimize the timing of key purchases.

Many retailers have fairly regular patterns of seasonal price changes and promotions, which are designed to maximize the lifetime profitability of a product. This is particularly important for fashion goods and high-tech, which tend to have a high initial price and a low clearance price. However, if customers (with the help of advisory services such as decide.com) start to game these price changes, then profit optimization becomes a lot harder to calculate. So this kind of advisory service represents a significant threat to retail profitability.

In September 2013, decide.com was acquired by eBay and effectively closed down. “This is an exciting opportunity to bring Decide’s expertise in data and predictive analytics to the worldwide commerce leader and empower over 25 million eBay sellers,” said Mike Fridgen, CEO of Decide.com. “We believe teaming up with eBay allows us to realize our mission of leveling the playing field in commerce.” (eBay 6 September 2013, Geekwire 6 September 2013)

In other words, taking the advantage away from the customers and giving it back to the sellers.

However, other customer-side predictive services may be still available, including GasPredictor.com (for Gasoline) and Kayak (for air travel).

Predictive Showrooming Dec 2012

Tricia Duryee, Decide.com Says It Will Accurately Predict Prices or Your Money Back (All Things D, 19 April 2012)

Thorin Klosowski, Kayak Adds Price Forecasting to Predict Price Drops and Increases (Lifehacker, 15 Jan 2013)]]> tag:blogger.com,1999:blog-6106782.post-3250489855449399072 On Customer Insight Richard Veryard https://plus.google.com/107891026040206160712 noreply@blogger.com 2014-02-26T14:40:00+00:00

On Customer Insight from Richard Veryard

]]>

On Customer Insight from Richard Veryard

]]> tag:blogger.com,1999:blog-6106782.post-4782162782935564907 Who owns data management strategy? Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2013-02-25T16:34:00+00:00

joel_schectman exposes an apparent divergence of opinion among #Gartner analysts - whether CEO or CIO should be in charge of data management strategy.

@ted_friedman says that taking out IT as the gatekeeper of centrally stored data can promote “better fact based decision making across the organization”.

@merv adrian says that bypassing the CIO can have unintended side effects like risks to privacy and the quality of the analysis.

Merv explains further “If you don’t have to go through a procurement process and IT, you’re a lot freer to do what you want,” said Mr. Adrian. “But all of that carefully constructed governance is completely undermined, you can be drawing incorrect conclusions, and exposing risks to privacy because they are doing things IT hasn’t vetted.”

Merv's concern about quality also applies to the widespread and often uncontrolled use of spreadsheets and other end-user tools. For example, we can find @JamesYKwak and @alexhern discussing whether we can blame Microsoft Excel for $9bn losses at JPMorgan?

What exactly do we mean by data management strategy? Joel says it includes how to best utilize customer information to leverage growth. Most CIOs seem to think their responsibility for data finishes when they deliver data and information to the user's device. They seem uninterested in how these users actually use the data, and whether better or faster data genuinely improve decisions and policies, and produce better business outcomes.

In other words, the CIO doesn't operate as a Chief Information Officer but as a Chief Information Systems and Technology Officer.

True information strategy includes a closed feedback and learning loop, so that the use of the information can be monitored. Are these expensively collected and elaborately processed data analytics actually influencing decisions, or are the users mostly ignoring them?

Alex Hern, Is Excel the most dangerous piece of software in the world? (New Statesman Feb 2013)

James Kwak, The Importance of Excel (Baseline Scenario Feb 2013)

Joel Schectman, Democratizing Data Analysis Has Risk (WSJ Feb 2013)

Updated 20 February 2016]]> tag:blogger.com,1999:blog-7415430.post-5050661576569029894 Using Analytics Correctly Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2013-01-25T22:54:00+00:00

gcharlton quotes a survey from @dbdsearch claiming that 80% of online retailers are using @GoogleAnalytics incorrectly (October 2012), via @FreshNick @hayden30.

Clearly Google wants online retailers to use all the features of the Google Analytics platform, which entails integrating with various other Google products and services (e.g. Google Adwords) as well as implementing all the necessary tracking codes and cookies according to Google's requirements. Any online retailer that fails to conform to Google's requirements is deemed to be using the platform incorrectly.

But what does "incorrectly" mean? Not doing what Google thinks you should be doing? Since when has Google been the ultimate arbiter of correct action?

We have been here many times before. There is often a significant gap between the designed product (how its designers expect it to be used) and the product-in-use (what the users actually do with the product). A designed product may have a number of sophisticated features that most users never get around to using, perhaps never actually need. On the other hand, the users trying to do a real job of work often display remarkable ingenuity in getting the stupid product to do something much more interesting than it was designed to do.

And sometimes there is a considerable delay until users discover the more sophisticated features. To cite a historical example, most early users of Lotus Notes used it as a substitute for technologies they already had, before they started to appreciate what it had really been designed for.

So there may be many ways people could learn to use Google Analytics better. As @haydens30 says, "there are basic best practice things that a lot of sites don't do - these are easy wins for any consultant".

And there may be many ways Google itself could make Analytics better and easier to use. In announcing some UI improvements yesterday, Nikhil Roy of the Google Analytics Team said "We hope you find these improvements useful and always feel free to let us know how we can make Analytics even more usable for you to get the information you need to take action faster."

What, we have to tell them? Don't they already know?
]]>

peteyMIT via @EthanZ explains how technology is changing the university admissions process.

When kids apply to university in the USA, it is becoming increasingly common to include a link with supplementary information about the applicant - for example a project tumblr, a YouTube video, a Flickr album of artwork. The links are typically coded to track visitors, giving the applicant some idea about the level of interest the universities are showing. Chris Peterson finds this an uncomfortable experience: "As admissions officers, we are accustomed to reading applications; now, applications are reading us. ... Applicants are now armed with unprecedented insight into the processes that decide their fate."

There are several problems with this. Applicants and their parents may be misled by the tracking signals collected by these digital supplements, which may yield an entirely false picture of the university process. And yet applicants may attempt to use these signals as evidence that an application has not been properly considered. Even if the university attempts to block the analytics, this may still send the wrong message. (The absence of a signal is still a signal.)

In the past, analytics were a tool used by large organizations to monitor and control their customers. We are now seeing analytic platforms that seem to allow customers to monitor and control large organizations. Large organizations now need to understand how much information they are exposing to these platforms, and what conclusions their customers may draw. We can expect similar examples to appear in many other sectors.

Chris Peterson, Opening the Black Box: Analytics and Admissions (Chronicle of Higher Education, January 2013)

Updated 25 June 2015]]>

UK Trialling App to Aid Government Decisions (BBC News 8 November 2012)

Here are some quick comments from Twitter

@lesteph PM's dashboard is at best pointless, at worst dangerous, unless his briefing system has fundamentally collapsed

@dominiccampbell he may as well have it, but pretending it's anything other than a partial view and mostly for PR is daft

@willperrin rather an antediluvian counsel of despair there then. back to 'ringbinders full of..' briefing

@6loss The "dashboard vs intelligence" debate? IMHO dashboards are useless without fast feedback on action.

In a subsequent discussion on Linked-In, @6loss and I discussed some of the intriguing questions raised by this news story.

Firstly, we were missing the imperative for real-time action and feedback. Obviously the Prime Minister needs to know whether job vacancies are going up or down, but the idea of real-time update is just ridiculous. Suppose that seventeen new job vacancies have been posted in Smartchester in the past twenty minutes, Are we supposed to believe that these seventeen vacancies urgently need to be communicated to the PM so that he can take appropriate action?

What does make sense is a dashboard that supports an OODA loop. A well-designed dashboard should not only provide aggregated data, but also provide some way of making sense of the data. (It is possible that the data visualization may help here.) And then taking rapid action.

But in a well-designed organization, the responsibility for rapid action is delegated to the people in the front line, who are given the real-time intelligence and the resources/tools and the authority to solve problems effectively and efficiently. This is what the military call "Power to the Edge". A completely different order of intelligence is required at the centre, usually operating at a much slower tempo.

And since managers are often tempted to meddle with randomly varying processes (Deming called this "tampering"), a well-designed control system deliberately hides much of the volatility from senior management. (In cybernetics, this is called "attenuation".)

Secondly, I'm wondering what kind of statistics we are talking about here. When people talk about "statistics", they often mean the kind of statistics kids learn in primary school (totals and averages) rather than the kind of statistics kids learn in high school (correlation and significance). I wonder how many ministers could cope with high school statistics (let alone degree level) without a civil servant or adviser there to explain it to them? The danger of the "dashboard" is that it may eliminate the vital step of interpretation and sense-making, which is surely essential to evidence-based management.

Thirdly, I'm wondering about the planned rollout of this App. Are we to suppose that all ministers and senior civil servants are going to be watching the same set of indicators, or does collective responsibility entail that each minister is watching a different set of indicators? In a typical control room, there are many people each watching a different dashboard or controlling a different sector: it would seem a bit redundant if they were all watching the same one. Meanwhile, the supervisor sits in his cubicle playing Angry Birds, or sending texts to his neighbours.

A few weeks after this discussion, writing in the New York Times, Will Wiles compared this dashboard with the Viable Systems Model implementation in Chile under Salvador Allende. He pointed out that the dashboard is not truly cybernetic because it lacks a mechanism to translate all that data into action. Quite so.

Will Wiles, Before Fruit Ninja, Cybernetics (New York Times, 30 November 2012)

]]>

UK Trialling App to Aid Government Decisions (BBC News 8 November 2012)

Will Wiles, Before Fruit Ninja, Cybernetics (New York Times, 30 November 2012)

]]> tag:blogger.com,1999:blog-7415430.post-206261083757385798 OrgIntelligence - Are Better Tools the Answer? Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2013-01-22T13:17:00+00:00

Information GatheringManagers spend up to two hours a day searching for information, and more than 50 percent of the information they obtain has no value to them. In addition, only half of all managers believe their companies do a good job in governing information distribution or have established adequate processes to determine what data each part of an organization needs.

Source: Accenture January 2007

The average interaction worker spends an estimated 28 percent of the workweek managing e-mail and nearly 20 percent looking for internal information or tracking down colleagues who can help with specific tasks.

Source: McKinsey July 2012

Knowledge Management

Traditional knowledge management has failed to address the problem of knowledge worker productivity. Tools that have been developed in KM focused on information management and do not support many of the key knowledge work processes. Knowledge workers have therefore adpated the email client to suit their needs. It has become the most successful knowledge work tool because it combines personal control with personalisability and integrates communication.

Source: Martin Roell, August 2004

Big Data

By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.

Source: McKinsey May 2011

Big data can create big value. But like all the big-data predecessors – i.e., databases, data warehousing, data mining, data analytics and business intelligence – you need to know what you’re looking for, why you’re looking for it, what’s it worth to you, and how will you take advantage of it BEFORE you start. Otherwise, big data will just be a big waste of money.

Source: Silver Peak Systems, May 2012

Social Media

Social media is addictive. And if you’re not too careful, it can seriously eat into your productivity.

Source: MediaBistro All-Twitter October 2012

Places are still available on my Organizational Intelligence Workshop (Feb 1st).]]>

Source: Accenture January 2007

Source: McKinsey July 2012

Knowledge Management

Source: Martin Roell, August 2004

Big Data

Source: McKinsey May 2011

Source: Silver Peak Systems, May 2012

Social Media

Social media is addictive. And if you’re not too careful, it can seriously eat into your productivity.

Source: MediaBistro All-Twitter October 2012

Places are still available on my Organizational Intelligence Workshop (Feb 1st).]]> tag:blogger.com,1999:blog-6106782.post-69447632226110878 What's Wrong with the Single Version of Truth Richard Veryard http://www.blogger.com/profile/04499123397533975655 noreply@blogger.com 2010-03-19T15:04:00+00:00

tonyrcollins reports, a confidential report currently in preparation on the NHS Summary Care Records (SCR) database will reveal serious flaws in a massively expensive database (Computer Weekly, March 2010). Well knock me down with a superbug, whoever would have guessed this might happen?

"The final report may conclude that the success of SCRs will depend on whether the NHS, Connecting for Health and the Department of Health can bridge the deep cultural and institutional divides that have so far characterised the NPfIT. It may also ask whether the government founded the SCR on an unrealistic assumption: that the centralised database could ever be a single source of truth."

There are several reasons to be ambivalent about the twin principles Single Version of Truth (SVOT) and Single Source of Truth (SSOT), and this kind of massive failure must worry even the most fervent advocates of these principles.

Don't get me wrong, I have served my time in countless projects trying to reduce the proliferation and fragmentation of data and information in large organizations, and I am well aware of the technical costs and business risks associated with data duplication. However, I have some serious concerns about the dogmatic way these principles are often interpreted and implemented, especially when this dogmatism results (as seems to be the case here) in a costly and embarrassing failure.

The first problem is that Single-Truth only works if you have absolute confidence in the quality of the data. In the SCR example, there is evidence that doctors simply don't trust the new system - and with good reason. There are errors and omissions in the summary records, and doctors prefer to double-check details of medications and allergies, rather than take the risk of relying on a single source.

The technical answer to this data quality problem is to implement rigorous data validation and cleansing routines, to make sure that the records are complete and accurate. But this would create more work for the GP practices uploading the data. Officials at the Department of Health fear that setting the standards of data quality too high would kill the scheme altogether. (And even the most rigorous quality standards would only reduce the number of errors, could never eliminate them altogether.)

There is a fundamental conflict of interest here between the providers of data and the consumers - even though these may be the same people - and between quality and quantity. If you measure the success of the scheme in terms of the number of records uploaded, then you are obviously going to get quantity at the expense of quality.

So the pusillanimous way out is to build a database with imperfect data, and defer the quality problem until later. That's what people have always done, and will continue to do, and the poor quality data will never ever get fixed.

The second problem is that even if perfectly complete and accurate data are possible, the validation and data cleansing step generally introduces some latency into the process, especially if you are operating a post-before-processing system (particularly relevant to environments such as military and healthcare where, for some strange reason, matters of life-and-death seem to take precedence over getting the paperwork right). So there is a design trade-off between two dimensions of quality - timeliness and accuracy. See my post on Joined-Up Healthcare.

The third problem is complexity. Data cleansing generally works by comparing each record with a fixed schema, which defines the expected structure and rules (metadata) to which each record must conform, so that any information that doesn't fit into this fixed schema will be barred or adjusted. Thus the richness of information will be attenuated, and useful and meaningful information may be filtered out. (See Jon Udell's piece on Object Data and the Procrustean Bed from March 2000. See also my presentation on SOA for Data Management.)

The final problem is that a single source of information represents a single source of failure. If something is really important, it is better to have two independent sources of information or intelligence, as I pointed out in my piece on Information Algebra. This follows Bateson's slogan that "two descriptions are better than one". Doctors using the SCR database appear to understand this aspect of real-world information better than the database designers.

It may be a very good idea to build an information service that provides improved access to patient information, for those who need this information. But if this information service is designed and implemented according to some simplistic dogma, then it isn't going to work properly.

Update. The Health Secretary has announced that NHS regulation will be based on a single version of the truth.

"in the future the chief inspector will ensure that there is a single version of the truth about how their hospitals are performing, not just on finance and targets, but on a single assessment that fully reflects what matters to patients"

Roger Taylor, Jeremy Hunt's dangerous belief in a single 'truth' about hospitals (Guardian 26 March 2013)

Updated 28 March 2013]]>

"The final report may conclude that the success of SCRs will depend on whether the NHS, Connecting for Health and the Department of Health can bridge the deep cultural and institutional divides that have so far characterised the NPfIT. It may also ask whether the government founded the SCR on an unrealistic assumption: that the centralised database could ever be a single source of truth."

Update. The Health Secretary has announced that NHS regulation will be based on a single version of the truth.

"in the future the chief inspector will ensure that there is a single version of the truth about how their hospitals are performing, not just on finance and targets, but on a single assessment that fully reflects what matters to patients"

Roger Taylor, Jeremy Hunt's dangerous belief in a single 'truth' about hospitals (Guardian 26 March 2013)

Updated 28 March 2013]]>