Data Management Strategies

The Algebra of Data Quality

2009-11-04T23:22:31-08:00

A basic algebra formula illustrates the value of data quality. To be sure, it is an illustration or a metaphor more than an actual formula for data quality, but it does demonstrate the need for and the value of information/data quality in enterprises today. The formula is a familiar one:

y = Ax² + Bx + C

In this illustration, "y" is the value that IT contributes to the business. "x" is the degree of data quality, ranging from 0 to 100. "A" is the money that the organization has put into BI systems. "B" is the money that the organization has put into operational systems. "C" is the money that the IT group spends managing complexity and building and running systems.

As you can see, if "x" (data quality) is zero, then the only value that the IT group contributes to the business comes from managing complexity and building and running systems. If x = 0, the value to the business of BI systems and operational systems (represented by "Ax²" and "Bx") are both zero.

You will also notice that when it comes to BI systems, the value is based on the square of the data quality (Ax²). If the data is of high quality, BI systems will yield highly valuable insight to the business. By contrast, if the data is of poor quality, the BI systems will produce nothing but garbage and will therefore be useless to the business.

There are many IT departments out there which are run by IT people who firmly believe that IT’s role is to manage complexity and to build and run systems for the business. In these IT organizations, data quality is not a high priority. As a result, the business derives little or no value from their BI systems and their operational systems. In these situations, the businesspeople are looking for ways to replace their IT groups, by adopting SaaS, cloud computing, and IT outsourcing. Clearly, IT people who believe that IT’s role is merely to manage complexity and to build and run systems are will find themselves competing unsuccessfully against external service providers who can do the job far more efficiently. That is not a good career path for IT people. On the other hand, data quality offers an excellent career path for IT people, because data quality is the key ingredient in the value that any IT group delivers to the business.

I recently read a fascinating article by a senior editor at Securities Industry News that illustrates how data management is no longer optional, especially for financial firms. Here is an excerpt:

… In fact, some firms may eventually decide to designate a chief data officer to manage a team of data specialists, such as data modelers, data stewards and taxonomists. The specialists, in turn, will figure out what data is stored where, verify its accuracy, build data models and create and attach metadata. The data officer and the team would decide who may access what data, integrate databases, and maintain archived data to meet regulatory requirements.

Most firms so far are maintaining the status quo -- keeping different pieces of data in many different databases.

Often, according to Atkin and other data experts, this can amount to as many as 30 different, somewhat redundant databases with discrepancies and errors, as well as 30 or more applications that consume reference data and incoming data feeds from over a dozen data vendors.

Such a hairball for collecting data just won't cut it any more.

"How a firm reacts to the need for accurate, timely, integrated data is now a critical differentiator in a competitive environment," warns Dayle Scher, a senior analyst with Tower Group, a Needham, Mass-based research shop. "Financial firms must be prepared to respond to increased due diligence on the part of investors and certainly more restrictive regulations looming."

After ensuring the data is accurate, says Scher, firms will need to share that data with downstream applications such as order management and portfolio accounting software and staff that need it, in the right formats and in a timely manner…

You can read the full article (entitled “Follow the Data When Tracking Systemic Risk”) here: http://www.information-management.com/news/data_management_risk_grc-10016449-1.html

I have discovered some excellent sources for eye-opening information on the need for data quality, including "Does Your Business Suffer From a Data Quality Reality Gap?", a YouTube "Information Management Fairy Tale", and www.IQTrainWrecks.com, a website dedicated to information/data quality disasters from around the world.

ROI on Data Quality and/or Data Management

2009-11-03T13:37:20-08:00

Blogger: Lyn Robison

We’ve had several clients ask us for help in defining the return-on-investment for data quality and/or data management for the benefit of their CIO. I struggle with that ROI calculation because it is so basic and obvious to me that I almost can’t articulate the value proposition. It is as though I am being asked, “Can you give me the ROI on doing regular oil changes on our company-owned vehicles?” I am afraid that I would find it difficult to come up with an ROI calculation for that one too.

The value of any information system can be determined by measuring the value of the information that that system provides to the business. If you neglect to do data quality and data management on any information system, the information within that system will eventually deteriorate into a worthless, indistinguishable blob of goo. If the information within an information system becomes worthless, then the information system itself becomes worthless too. Any information system that provides useless information is useless. Like an airplane that can’t fly, it is unfit for its intended purpose.

What is the ROI on not letting a corporation’s valuable assets, such as its information systems, deteriorate? I guess you show that if you neglect any valuable asset, its value will deteriorate to nothing. So, the inputs are the value of the asset today (Vi), and the value of the asset after one year of neglect (Vf). Then to calculate the ROI, (or in this case the yield, effective interest rate, effective annual rate, or annual percentage yield) you take (Vf – Vi)/Vi. As you can see, if Vf is higher than Vi, you will get a positive yield. If Vf is less than Vi (if the asset’s value is deteriorating) you will get a negative yield.

To do this calculation, you have to be able to quantify Vf and Vi. Coming up with precise, defensible numbers for Vf and Vi can be difficult. And as a result, many CIOs have thought (consciously or unconsciously), “Hmmm, there is no ROI for data quality or data management.” As a result, those CIOs have not prioritized data quality or data management work. Instead, they are passively letting the value of their existing information systems deteriorate to nothing. That approach does not make a lot of sense to me.

This is where information quality metrics come in handy. If an enterprise were to measure the quality (a.k.a. "the usefulness to the business") of the information coming from their information systems over a one-year period, they could calculate the yield (positive or negative) in the value of their information systems. If the IT organization can keep the yield positive, I guarantee that the business will be pleased with the IT organization’s efforts. I also guarantee that if the yield is negative (whether it is actually measured or not), the business will not be pleased with their IT people and will look for any opportunity to outsource or replace them.

Enterprise IT: The Chinese Finger Trap

2009-10-26T20:10:41-07:00

Blogger: Lyn Robison

You’ve probably encountered the Chinese Finger Trap, a puzzle that traps the victim's fingers in both ends of a small, woven cylinder. The intuitive reaction is to pull your fingers out of the ends, but this only tightens the trap. The more you try to do the intuitive thing, the worse your situation gets. The solution is counterintuitive: push the ends inward toward the middle.

It turns out Enterprise IT is a lot like that. IT people often find themselves trapped in a situation where doing the intuitive thing only makes things worse. The business asks for an IT system, so the IT people work hard to implement the system, but then the businesspeople end up not being satisfied with it. And then the businesspeople want more systems, so the IT groups build more systems just as fast as they can. The businesspeople always seem to want the systems faster than IT can implement them, and then the businesspeople seem to be always dissatisfied with the results -- and it is all IT’s fault. That’s the finger trap. The more that IT groups try to give the businesspeople the systems that they ask for, the less satisfied the businesspeople seem to be.

The solution is counter-intuitive. When businesspeople ask for a new system, what they are actually asking for is information. That is why they want the new system -- to get some information that they need to do their jobs better. So, when IT delivers a new, hurriedly-built system, these new systems often don’t quite deliver the information that the businesspeople hoped for, the businesspeople are unsatisfied. And then they ask for another system. The solution, the way out of the trap, is to stop building more systems. Really.

As counterintuitive as this sounds, the solution is to take a careful look at the information that the businesspeople require, and see if there is a way to clean up the data in a few of the old systems to give the businesspeople the information they need. Cleaning up the data in existing systems is often faster, less expensive, and more effective than implementing a whole new system. The businesspeople will like it better and you will escape from the trap of working harder while satisfying the business less.

Is BI destined to die like SOA?

2009-10-20T17:18:25-07:00

Blogger: Lyn Robison

Business Intelligence is a hot topic. Enterprises are spending large sums on BI initiatives. Unfortunately, BI projects are failing more than they are succeeding. BI initiatives typically do not deliver their promised benefits. I just saw a survey that says almost two-thirds of companies that employ BI are being barraged with complaints that the system isn¹t doing what they need it to do. Another survey undertaken by the National Computing Centre found that only 13% of BI projects undertaken in the United Kingdom lived up to expectations. This figure is mirrored by a number of other studies.

We at Burton Group are coming out with some guidance soon that should improve your odds of success on your BI projects. In a few days we will publish a paper entitled, “The BI Iceberg: It’s What’s Beneath the Surface That Matters”. And within the next couple of months we will publish a BI maturity model that will help you evaluate the level of maturity (and by extension, the chances of BI success) in your enterprise. Maturity models are helpful because they are both descriptive and prescriptive. They are descriptive in that they help you see where you are. And they are prescriptive in that they help you see what you need to do next to make progress.

Having said this, however, I am concerned about BI’s future. With these high failure rates, BI seems to be on par with SOA. My friend and colleague Anne Thomas Manes wrote an obituary for SOA (here). I have to wonder, in a few years will someone be writing a similar obituary for BI? From what I can see, if we in enterprise IT don’t find ways to make BI projects more successful more often, BI is likely to die and take the careers of many BI professionals with it.

There is an alternate path: one that leads BI to success and prosperity. This alternative path runs counter to conventional thinking about how to implement BI systems. This alternative path to BI success truly is the road less traveled, and leads beyond business intelligence to reach business insight.

Professionals who embark on this alternate path understand that traditional BI is based on a false notion. The false notion is that it is feasible to take applications that were built to address localized requirements and stitch their data together (using data warehouses) into a larger picture that represents reality at the enterprise level. In the first place, the data from different apps won’t fit together. All of the king’s horses and all of the king’s men can’t put Humpty Dumpty’s data together again, because the data from individual apps was never designed to fit together in the first place. And in the second place, the chronic lack of data quality means that most application data tends to contain garbage anyway -– and garbage in, garbage out. This false notion that all you have to do in BI is pull together data from various apps is no doubt at the root of the high failure rates of BI projects. It usually can’t be done. And as the high failure rates attest, traditional BI simply doesn’t work.

The first step in this alternate path that leads to BI success involves understanding the relevant data. You have to understand what the data means, what it represents, how closely it reflects reality (its data quality), what part of the total enterprise picture it paints, what localized or provincial assumptions mar the data, who governs the data, how does the data and the part of the business it represents relate to other pieces of data and the parts of the business they represent, etc. In short, this alternative path to BI success requires BI professionals to switch from being software implementers to being enterprise data people.

BI systems are not the engine and data merely the fuel. Rather, BI systems are merely the mine shafts and the data is the gold. In this alternate path to BI success, you understand, optimize, and cater to the data, and the data is all but your exclusive focus. When BI professionals focus on building the data instead of on building BI systems, the systems they implement go beyond business intelligence – they yield valuable business insight, and that is BI success.

We at Burton Group have published lots of guidance and will publish further guidance in the future on how to treat enterprise data like gold. My hope is that we can save BI from the painful death that SOA experienced, and turn BI from Business Intelligence into Business Insight.

Enterprise Data: Byproduct of IT Systems, or Business Asset?

2009-10-13T15:33:49-07:00

Blogger: Lyn Robison

I like to ask IT professionals the following question. What exactly is an IT department’s contribution to the goals of the enterprise it serves? (I am not talking in the abstract here – an answer of “better business results” is too abstract – I want to know what exactly does the IT department deliver that results in better business results?)

Anyone whose answer is “information systems” is tightly coupling two distinct concepts: information and systems. Systems are the apparatus in and through which information is processed and conveyed. Information and systems are distinct concepts. It is quite possible to deliver systems with no useful information (which I have seen IT groups do all too often). It is also quite possible to deliver useful information using a variety of different systems, from an abacas to a heads-up display.

I can see absolutely no benefit from, and only danger in, tightly coupling the two distinct (albeit related) concepts of information and systems. One is the end and the other is the means. Information is the product and the systems are the factory. By blurring their distinction, one can easily forget which is the end and which is the means.

Many IT professionals have indeed forgotten which is the end and which is the means. They have come to believe that systems are the end-product of their work. For these IT professionals, it is easy to believe that data is merely a byproduct of systems. Data is that pesky cruft that builds up in the database when a system has been in use for a while.

This is dangerous thinking. This thinking will eventually do great harm to the careers of many IT professionals. Systems are not the end-product and data the byproduct. In truth, systems are the factory and information is the end-product. Information is an asset that is expensive to produce and is highly valuable when managed competently and used properly.

The business does not want systems that produce data cruft. They want systems that produce useful information. The business doesn't want systems that produce data cruft, and the business can no longer afford IT professionals who implement systems that produce data cruft.

So, if IT professionals continue to believe that the systems are the end-product of their work and that data is merely a byproduct of those systems, they are bound to find themselves without a job.

A Report From the 2009 Ig Nobel Awards Ceremony

2009-10-07T04:56:35-07:00

Blogger: Joe Maguire

Last Thursday (01 October 2009) I participated in “The Nineteenth First Annual Ig Nobel Prize Ceremony.”

Before the ceremony I worked backstage. This mostly involved preventing prize winners awaiting the ceremony from wandering into the remote recesses of the basement of the Sanders Theater on the Harvard University campus. Another chore: securing the signatures of the attending Nobel Prize winners upon the certificates presented to this year’s winners of the Ig Nobel Prizes.

Once the ceremony began, my chores were over. As I sat back to enjoy the proceedings, I began to appreciate this year’s honorees as only a data-management professional could.

A previously overlooked advantage of semantically rich identifiers:

The veterinary medicine prize went to Catherine Douglas and Peter Rowlinson of Newcastle University, Newcastle-Upon-Tyne, UK, for showing that cows with names give more milk than nameless cows. If you’re scoffing at the existence of a prize for veterinary medicine, well, you’re probably the cold-hearted sort who distinguishes cows by COW_ID or some other method that denies the essential dignity of our bovine friends.

Data analytics is only as good as the underlying data:

The literature prize went to Ireland’s National Police Service (An Garda Siochana), for identifying the worst driver in the country, whose name—according to the databases maintained by the police service—is “Prawo Jazdy.” Prawo Jazdy is Polish for “Driver’s License.”

Some questions will require the collection of new data sets:

The peace prize went to Stephan Bolliger, Steffen Ross, Lars Oesterhelweg, Michael Thali and Beat Kneubuehl of the University of Bern, Switzerland, for determining — by experiment — whether it is better to be smashed over the head with a full or empty bottle of beer.

The ceremony included other highlights, including acceptance speeches, which were subject to a one-minute maximum (charmingly enforced by an eight-year-old girl), and a handful of speeches by leading thinkers in various fields, which were subject to even stricter limits. These speeches are called “the 24/7 speeches” and they work like this: A prominent thinker or world authority has 24 seconds (strictly and uncharmingly enforced by an imposing fellow with a stopwatch and a whistle) to describe his or her field in technical jargon. Then, the speaker must paraphrase his or her own techno-babble in exactly seven words that anyone can understand.

Paul Krugman, Op-Ed columnist for the New York Times and winner of the 2008 Nobel Memorial Prize in Economic Sciences, gave one of the 24/7 speeches. The second component of Dr. Krugman’s speech contained eight words, not seven. You may supply your own quip about the precision of the economic sciences here. (And you can read Krugman's own blog entry about the Ig Nobel prizes here.)

Marc Abrahams, the driving force behind the Ig Nobel Prizes and the editor of Annals of Improbable Research, says that the awards are intended to recognize achievements that “first make you laugh, then make you think.” For those of us inclined to think about data management, these awards provide a rich, albeit smart-alecky, opportunity to contemplate some of our most vexing problems about data semantics, data quality, data precision, data analytics, and getting smashed on the noggin by beer bottles.

Despite the light-hearted tone, the questions deserve attention:

Are arbitrary identifiers (such as Cow_24563) better or worse than semantically rich identifiers (such as Elsie)?

Here, data analysts would be wise to work with business users to set the policy on a case-by-case basis. Although arbitrary identifiers are often convenient in programming and implementation contexts, they should never be used in a way that prevents users from maintaining business-motivated, semantically rich identifiers.

How good is the underlying data that supports analytics?

Here, data analysts should think about data quality (which is not the same thing as software quality!) in a formal way that is buttressed by data-governance, master-data-management, and data-quality-metrics initiatives. The global economy doesn't make this any easier: "Prawo Jazdy" will soon be Irish slang for "I'm not paying this traffic fine."

What happens when new data sets are required?

Here, data analysts should tread lightly. Situations that require entirely new data sets (such as comparative impact tests with full and empty beer bottles) are atypical. Most data is under-analyzed, and the typical business can be well-served by an IT organization that first seeks to analyze existing data before creating more data.

Attending the Ig Nobel ceremony is always fun, and working backstage offers special perks. This year, I met seven Nobel laureates. Maybe it was eight. Whatever.

Internet Scale and Map Reduce

2009-10-04T16:44:56-07:00

Blogger: Marcus Collins

My colleague Lyn Robson recently blogged about the emerging database meta models. Lyn mentioned the term Internet Scale in the context of data processing. This term is used to cover conditions where the traditional relational model is often not the correct choice and Map Reduce maybe more appropriate.

Organizations should consider a wider definition of internet scale as they ask when Map Reduce should be considered. Internet scale should include:

Data volumes – huge data volumes is still a key criteria for internet scale. The first examples of these were web clickstream logs and these are still an important data source. These data volumes are seen in other domains for example security logs for intrusion detection, credit card transactions for fraud detection etc.
Complex Processing – for all its power SQL lacks the ability to perform complex data processing for example statistical analysis.
Semi-consistent data – the Map Reduce programming model is similar to the relational full table scan. As such semi-consistent input data is important to such processing. I’ll explore this more in an upcoming paper on Cloud Databases.

The traditional database vendors have not missed how important Map Reduce is becoming. In a recently published document "Data Warehouses: Navigating the Maze of Technical Options" Burton Group Senior Analyst Marcus Collins looks at the differing database technologies that database architects have to choose as they design their data warehouse.

One of these is Map Reduce which is a programming model first developed by Google in 2004 for the processing of large data sets. Whilst not a database technology per se, the framework is included in this overview because of the coverage this framework has received and the work currently underway to integrate database access into the framework. Map Reduce consists of 2 functions map and reduce and a framework for running a large number of instances of these programs on commodity hardware. The map function reads a set of records from an input file processes these records and outputs a set of intermediate records. These output records taken the generic form of (key, data). As part of the map function a split function distributed the intermediate records across many buckets using a hash function. The reduce function then processes the intermediate records. Both the map and reduce functions can be written in any programming language.

As an example of Map Reduce – the requirement is to count the number of words that occur within a document or set of documents. The map function splits each document into words and outputs each word together with the digit “1”. The output records are therefore of the form (word, 1). The Map Reduce framework groups all the records with the same key (i.e., word) and feeds them into the reduce function. The reduce function sums the input values and outputs the word and the total number of occurrences in the document(s).

Both the map and reduce functions can be parallelized thus providing the scalability required when the input data is internet scale. This example shows the similarity between a parallel full table scan and the map function and it is this synergy that is being exploited by database vendors.

The primary benefit of the Map Reduce framework is the flexibility it allows in the type of data that can be access and the processing within the map and reduce functions. The map function can read from standard flat files (e.g., web clickstream logs), web crawls and database tables. There is no limit on the programming language or type of processing that the map function can perform. For example the map function could be written in Perl of Python and perform unstructured text analysis or statistical analysis. This flexibility should be compare with the limited processing supported by the SQL language even when we include the increased sophistication provided by the major database vendors (e.g., stored procedures or functions).

There are a number of drawbacks with Map Reduce. The records processed and written by Map Reduce have no schema – taking the generic form (key, data). Schemas provide documentation of the record structure, provide independence from schema changes through the use of views and require that data must adhere to the schema definition thereby guarding against corrupt data. Map Reduce’s lack of schema support provides none of these capabilities. Map Reduce does not make use of any indexing schema. Rather the map function uses brute force and massive parallelism to provide the necessary processing throughput.

The Burton Group document "Data Warehouses: Navigating the Maze of Technical Options" covers a number of other database technologies and I’ll explore these in a series of follow on blogs.

Future of the DBMS, considering Hadoop, NoSQL, and XQuery

2009-09-29T21:21:30-07:00

Blogger: Lyn Robison

In Burton Group’s recent “2010 Planning Guide: Data Management Strategies” paper, I said, “the data management foundation that [relational] DBMSs provide is not adequate for all of the needs of modern enterprises.” Clearly, I believe the era in which enterprises use relational database servers to store all of their data is nearing an end.

Today I listened to a briefing from a company that is betting its future on “internet-scale data processing” which Hadoop offers. I imagine that this company will do fine, but it appears to me that Hadoop and NoSQL databases will not supplant relational database servers in enterprises until there are clear standards for client data access and processing. The widespread adoption of Structured Query Language is largely what made relational databases so popular. SQL made it possible for clients (both humans and software applications) to easily process and retrieve data in relational data stores. Until we see the widespread adoption of some programming metaphor or language for client data access and data processing in the NoSQL world, NoSQL databases are likely to remain niche products.

Does this mean that enterprises are stuck forever in the relational-only world of DBMSs? Not necessarily, because in the NoSQL world there is no equivalent to SQL … except, of course, XQuery.

XQuery is poised to do for XML databases what SQL did for relational databases. Using XQuery, clients (both humans and software applications) can easily process and retrieve data in XML data stores. This means that developers can use XQuery with their favorite object-oriented programming language to build XML-based applications. Application developers now have a choice for their data storage needs that enables them to select the meta-model that best fits their data. The relational meta-model is best for tabular, structured data, while the XML meta-model is best for document-centric, semi-structured data.

Like I said, developers can use object-oriented programming languages with XQuery to build XML-based applications –- that is, if they want to build those XML-based applications the wrong way and write a bunch of unnecessary software. In one vital sense, XML data is fundamentally more powerful than relational data: XML data does not need to be instantiated as objects in OO code in order to be processed and presented to users. Instead, XML data can be decorated and enriched with tags, and these tags can be processed and understood by software that enterprise IT groups do not have to write. If an enterprise’s data is in XML, lots of off-the-shelf, well-tested, well-proven, enterprise-ready software packages can process, display, and edit that data.

The software applications that can understand XML tags include Microsoft Word, Excel, InfoPath, as well as the Open Office productivity applications. IOW, enterprises can now use Office apps, including Word, Excel, and InfoPath for the user interface for presenting and editing enterprise data, provided that data is in XML -- more specifically, provided that data is in the native XML file format of one of the Office applications. But how do you get enterprise data into the native XML file format of Word, Excel, or InfoPath? You use an XML database such as MarkLogic Server that understands and that can transform enterprise data into and out of those Office XML file formats.

The vendors of relational database servers have not yet made their DBMSs capable of transforming enterprise data into and out of those Office XML file formats, which shows that they are somewhat out of touch with power of XML-based enterprise data. At least the relational database vendors have been smart enough to make their DBMSs expose relational data as XML, which means that application developers can choose the right meta-model (relational for tabular data and XML for semi-structured data) and then can treat all of that data as if it is XML. By treating all data as if it is XML, developers can finally say goodbye to the ever-troublesome object-relational impedance mismatch.

These are radical ideas that can take solution delivery in enterprise IT to new heights of effectiveness. These ideas will no doubt be viewed as heresy in certain circles. I have published my heretical ideas in a document entitled “The Methodology for Overcoming Data Silos (MODS): Using the New XQuery Development Stack”. In this document, I specify an application development stack that is built around XML- and XQuery-based development tools. Give it a read and let me know what you think.

Fix What Ails Your IT Group: Focus on Information

2009-09-28T09:02:24-07:00

Blogger: Lyn Robison

Something is not right in enterprise IT. Corporations spend piles of money on enterprise IT, but business leaders do not seem satisfied with the work of their IT groups. There is a simple antidote. A focus on information will fix what ails enterprise IT. Businesses pay for IT systems not because they want technology, but because they want information. If enterprise IT groups would focus on delivering information (instead of just delivering systems and technology), the business would at last be pleased with IT’s work.

Sound too simple? Let’s examine the alternatives.

Without a focus on information, IT’s work on services and architecture turns into activities that benefit IT instead of the business.

Without a focus on information, IT’s business process automation efforts turn into rigid silos that often don’t fit the business processes.

Without a focus on information, the BI systems that IT builds turn into Business Stupidity (BS) that cannot answer businesspeople’s questions.

Without a focus on information, IT agility turns into yet more useless IT systems delivered faster and cheaper.

Without a focus on information, IT’s focus on business outcomes heightens the risk of unemployment for IT people.

The simple fact is that if an IT group does not focus on delivering useful information to businesspeople, nothing else that the IT group does will work.

The place to start is IT metrics. The first step in focusing on information is to establish information quality metrics as the primary metrics by which an IT department measures its work. For guidance on information quality metrics, click here.

One Way for IT to Satisfy the Business

2009-09-14T11:18:22-07:00

Blogger: Lyn Robison

Imagine a businessperson saying, “I want to see all of the information that exists (and that I am authorized to see) in all of our company’s systems about this particular customer. That includes transactions, contracts, correspondence, payments, shipments, complaints, history, everything.”

As an enterprise IT person, what would be your reaction?

1. “That’s impossible. I mean, good luck. You are naïve to think that diverse information can be meaningfully integrated from that many disparate information systems. It would take forever to hard code the logic to translate all of that information from its original context into the context that you are requesting. ”

2. “That’s not needed. No one ever asks for that. There are no business scenarios where finding all of the existent information on a particular topic would ever be needed. And besides, the purpose of IT systems is to automate and optimize business processes. IT systems do not ever need to provide impromptu access to information.”

3. “That’s what data warehouses and BI apps are for. We’ll just create a data warehouse and BI application for you. Of course, that will only give you access to the structured data. And, if you want to ask a follow-up question on related topic, we will probably have to create yet another data warehouse and BI application for you. BTW, how much money can you put into this and how long can you wait?”

4. “Can do. You can see what every system in the enterprise has on each customer by visiting our MODS registry with your browser or favorite Office application. The source system for each piece of data will be clearly shown so that you can see the context that the data is coming from. The contact info for the business data stewards is included with each piece of information, so you can get answers to any questions you have about the meaning of the data from each system.”

Now imagine that the businessperson says, “How reliable is this information?” What would be your reaction?

1. “I am sure it’s good enough. It’s probably not 100% reliable, but it’s better than nothing, right?”

2. “Couldn’t tell ya. The data might be wrong, but that’s not really my problem. I only deliver the technology and manage the systems.”

3. “It’s not black and white. The data is not reliable or unreliable. It’s somewhere between completely erroneous and totally true.”

4. “Included with the information from the MODS registry is a link to the information quality metrics for the data from each system. This will give you measures of the reliability of the data from the businesspeople who know it best, and will also give you the lineage of the data from each system.”

What answers would the businessperson prefer to hear from you? Based on my own experience, the answer 4s are going to be the preferred answers.

Many may think that the answer 4s would be impossible, or impossibly expensive to deliver. I respectfully disagree. The technology to do this has existed for several years. The data is already lying around the enterprise. MODS projects are inexpensive. Information quality metrics are easy to institute. And data stewardship and governance are not onerous to implement, and with the demands of regulation and compliance, are going to be essential anyway. Do those things and you will be able to give answer 4 to both questions, which are answers that the businesspeople will find quite satisfactory.

It is interesting to note that these “find all the info on customer X” questions get asked and answered every day. This is why CEOs and EVPs have executive assistants (otherwise known as poor go-fers). The CEO says, “I have a big meeting with customer X in 3 days and I need to be armed with all our interactions with them.” With that, the go-fer whips open the BI dashboard, does a query for the systems he or she can get at, calls Operations and has them do a scan of archived e-mails from the past 6 months, reads and summarizes those, logs on to every search system the company has, and does a query for customer X. After three 12-hour days, the go-fer hands the CEO a printed out Word document with everything neatly formatted. So it isn’t so much that the question can’t be answered, it’s that the question could be answered in a much faster and higher quality way if the IT systems were changed.

In short, assuming a company has the labor to hand-stitch the information together, I’m not sure high level management recognizes this as a problem. Their questions get asked and answered. The issue is that only CEOs and EVPs can ask these questions; everyone else is out of luck. A MODS registry makes the information readily avialable to everyone who is authorized to access it.

Ultimately, the MODS registry could be made transparent to businesspeople. The businessperson could use a simple query or search tool to submit a request for complete customer information. That request could automatically route (e.g., via a data services platform) to the MODS registry server. The MODS registry system could resolve the query or search by accessing registry data and business logic. The logic would resolve duplicative, inconsistent, and conflicting customer information from multiple sources. The resultant solution to the query (i.e., a complete view of a customer) could appear in a window on the businessperson’s access device regardless of its location (i.e., assuming appropriate access and security controls). You can learn more about MODS by clicking here.