Monash University Business Intelligence Blog

Do we really build systems to improve decision making?

2009-11-10T10:23:00.005+11:00

Just wrapping my mind around a new direction for a research project. Some of you know, I've been around the traps in the last little bit doing my schtick on interfaces. Of course everybody says the interface is important but I don't think we do a lot about it.Main message of my talk is that we don't devote enough effort or resources to the design of the interface. We just use the "orthodox" tables and charts provided by the vendors in our data displays (and the same for the navigation between views of data) - and many of them are not very good.

We just did a study where we can two different groups of people the same data - but one group got the data in "orthodox" BI charts, complete with 3D effects or gradient shading, not over the top but typical of BI systems today - and the other group an equivalent set of charts - very plain in their design (think the design principles of Tufte or Few). As we hypothesised, when asked questions about the data that required accurate analysis - the group that got the plain charts did much better than the group that got the sexier charts. Neat to see that this study and article that Trevor Clarke wrote for Computerworld after a chat with me has stirred up some debate.

Most BI systems provide users with a slicer dicer style interface in the hope that they will explore the information our systems give them access to - finding the "number" that solves their problem - just like in the demo's the vendors give. Sadly for that dream, in another study, we showed that a lot of people can't use pivot tables.

I think we have a problem. A serious one. Nigel Pendse thinks (based on his global survey) that the mean usage rate of a BI system is around 7 to 8%. That seems a little low to me - but its been a while since we collected that kind of data, and with web technologies (esp. in the last couple of years) meaning that more people have access to BI systems, the proportion using them might have fallen - so while 7-8% seems low, his data collection is good, I've got to think that he must be close to right. Even if he's not, the best take up rates we have seen in our case study work aren't very good.

In my presentations and writing I've been arguing that one main reason for this low usage rate is the interface we provide. I think that most BI systems have very similar interfaces (with few exceptions, the character of the main offerings from the BI vendors is identical). Those interfaces suit a few people, but not many. We need a major re-focus on the role and place of the interface if we are going to get better rates of usage of BI systems.

Now, there are a couple of underlying assumptions to those assertions. The first is that its desirable for lots of people to have access to a BI system, and to use that in their daily work. That may not be true. There have been studies that show that using decision support systems actually decreases decision performance - so the people using BI systems might actually be doing their organisations and themselves harm. I'm happy for the moment to believe in the value of BI systems, and leave the research on that one to other people.

The second assumption that I have been thinking about - is that people use BI system to help make decisions. What if that's not true? What if people use these systems for some other purpose. It could be (as would be predicted by cognitive artefacts like the confirmation bias) that our systems are to build a case to convince ourselves and others that a decision already made is correct?

If that's the case, no wonder we are putting 3d funnel charts in our systems - in response to user demand. The aim of the display of data would be to impress people, show them how clever we are, how the data supports our strategy (better obscure the data a bit just in-case it doesn't). Im exaggerating the case a little (of course), but it is interesting to think about - and not to hard I think to design a research instrument that can be used to go out and find out what BI systems are actually being used for: decision making or decision justification?

POD

:-( back to marking exam papers.

Timo Elliott's demonstration dashboard

2009-06-21T23:23:00.005+10:00

Neat little demo put together by Timo Elliott (@timoelliott) from Business Objects using their Xcelcius product as a - think I'm inventing words here - mashupable object. He plonked the dashboard he made, on his blog with a number of controls that allow you - YouTube style - to share via email, twitter or cut and past as an object in html. This kind of functionality will get even easier when HTML 5 gains traction, but as Timo shows its do-able right now. Doesn't matter that it's a dashboard, the idea could work with any report or report part. I really like the ability to click to expand to full-screen view - just link clicking on a web page section in iPhone's Safari, a visual drill-down (or up).

So with a click on Timo's blog and a paste, here is Timo's demo Dashboard (his blog has some context that's fun to read too). To start with our blogger hosted blog isn't coping with the width of the element so well, but Timo kindly set a new code block that does fit - Thanks Timo.

An experiment on Twitter

2009-06-10T09:19:00.007+10:00

For a long time a few of us, and not just at Monash, have wondered how a real time text feed - perhaps in RSS format - might be applied to BI systems. Now that social networking sites like Facebook/Twitter/Friendfeed exist and are becoming widely used the idea has a bit more traction with people we talk too - nothing like concrete experience to help people understand what you are talking about.

I've just set up a Twitter account that I'm going to use to develop a prototype system to demonstrate how a "feed" of text updates might be useful in a BI context. This feed (@monashbiindex) will contain updates and observations on data collected as part of out BI Index project.

Right now, all it will report on is a single number - the number of on-line job ads placed in Australia for positions related to business intelligence and data warehousing. I've been collecting this number (off and on) since 2005. It's quite interesting, there are weekly wobbles (up on Thursday and Friday) and significant seasonal variations too (up before and after the end of the financial year - way down in January). The analysis we are doing will soon expand (just like any dimensional data set) to include more detail like locations, industry sectors, job categories and the like. Later we might extend it to other countries but right now we'll stick to Australia.

So, I've got a nice little automated app, that will at 9:00 Am each morning go to seek.com.au, do a search and grab from the resulting page the total number of jobs. It records that number in a database - along with the data and time. Then a 'reporting' app fires up and does a simple day by day comparison of the number to the previous days and posts a tweet. Much better than the Excel macro's I've been messing with since 2005! Once I'm happy with how that's working, I'll extend the range of topics tweeted on to include a wider range of temporal tweets (end of week, end of month, end of season summaries), link the tweet to reports (no 3D donut charts I promise) and start to build agent style data monitors that look for exceptions, to demonstrate how a twitter style feed might be used for reporting the results of data mining algorithms.

POD

P.S. Oh, later the "Index" will include more measures of health than job ads. We are planning a regular survey of the "industry" and also a stock market index - something again I started a long time ago.

P.P.S The BI Job index is based on the number of jobs that match the search terms "Business Intelligence" or "Data Warehous" that are listed on www.seek.com.au. The index is expressed on points based on a ratio of the number of jobs to the number on the day the index started 23/10/2005. On that day there were 349 jobs - 100 index points.

Light at the end of the tunnel

2009-05-31T21:01:00.004+10:00

Rob and I and co. bloggers have been more than a little quiet as - despite our best intentions - we got run over by our teaching work load in semester 1. Semester 1 is about to end so we'll get back to some BI blogging very soon.

Worth mentioning that we'll also be starting a BI podcast (not the same as the ones we have for our units). This will feature interviews with BI "thinkers" from Melbourne and around the world, as well as the occasional talk from members of the Centre. We will start with a couple of talks from me - the first will be a "briefing" on the decision support industry I gave to Prof. Arnott's FIT5094 class a fortnight ago, and my closing keynote at the Mastering Business Objects conference in Sydney last week. We hope to have a new posting in the podcast stream every two weeks. If you have any ideas about people you'd like to interview, or topics you'd like us to cover, please let us know.

POD

Cranky Geek - John Dvorak - takes huge swipe at spreadsheets, BI and accountants.

2009-01-23T14:13:00.004+11:00

Lots of other things I want to and should post at the moment, but I couldn't let this slip. An article by John Dvorak that is kicking up quite a storm. Love John Dvorak. Always worth reading and listening. He's often wrong (and very wrong), but always there is something to what he says. Anyway, he is an article about the 30th anniversary of the spreadsheet.

He's nuts of course (in a good way, and that's what I like about him), but he makes a point. We have all this wonderful what-if analysis, information at our figure-tips, enterprise Bi all over the place - so how come we aren't making better decisions? Of course, the spreadsheet isn't to blame but its a fun read.

The 30th Anniversary of the (No Good) Spreadsheet App

POD

Scoble on BI Panorama/Google style

2009-01-22T11:48:00.003+11:00

Thought it was worth drawing your attention to a recent video post by Robert Scoble.

The story of 2009? Enterprise disruption?

It's not particularly critical - really just a PR puff piece - it covers the tool's Panorama have been creating in partnership with Google. Panorama are a company to watch, Novaview - their core offering - is excellent, and they are the company that sold Microsoft the technology that became Analysis Services.

However, I'm underwhelmed by the tools they have created with Google so far, but its a start I guess, into a potentially interesting shift of BI services in the 'cloud'.

Have a watch, I'd be interested in your thoughts. I'll post a little later on why I think this in general is a big deal, but this particular tool isn't.

POD

Simple Designs are Hard

2008-12-03T12:31:00.004+11:00

Reading Peter's last couple of posts got me thinking about a great TED conference presentation made by former Broadway pianist and NY Times tech columnist David Pogue. A couple of years ago he talked at TED about the design of technology and the importance of simplicity in design.

A lot of what we do as BI developers is design ways for people to mess around with, and learn from, information. One one of the key principles behind a doing this well is to ensure simplicity. A lot of what Tufte and other data visualisation experts talk about can be seen to derive from this principle, and as Peter said, despite the experimental and anecdotal evidence to support it, it's something the vendors often don't do well. One of the reasons for this is that it's just plain hard, and probably something that most software engineers are not very good at doing. Simple, elegant and intuitive interfaces for BI apps are not just aesthetically pleasing, they lead to better understanding on the part of decision makers, and creative uses of the tools that can lead to unexpected insights - which sounds awfully like the vendors' own jargon. I wish they'd listen to people like David Pogue a bit more.

An unexpected finding ...

2008-12-03T11:30:00.006+11:00

It's a good time of year to be an academic, nearly all the marking and teaching related administration is done for the year, though next year is approaching fast - we do have some time to fully devote our attention to research. We have a lot of projects that are finishing up, which means it time for us to get out and start collecting data for the next round of case studies and investigations. We are also doing some tiding up of our infrastructure. We have had to move out of a room we had devoted to project related activities but that has given us an chance to throw some stuff out and generally get our "house" in order. For example we been have updating and sorting out our files hosted on various servers. None of that has any direct impact on this blog, except we have run out of excuses not to extend our blog related activities a little. Shortly, we'll have a podcast featuring presentations and interviews by and with staff from the Centre. Another thing we will start to do is talk a bit more here on the blog about our published research. So lets start that right now ...

Here is a link to a paper we published earlier this year.

A note on an experimental study of DSS and forecasting exponential growth

(The file is hosted on Science Direct and they own the copyright, so sorry if you can't access it. If you are on the Monash network, you'll be able to view it, or if you have a Monash authcate, try using the VPN. If you are at another Uni. you'll probably have a subscription)

I know that sounds a bit technical, and that its not of interest if you aren't into forecasting or that worried about exponential growth, but actually, its interesting beyond those areas. The paper presents and experiment we conducted where we asked subjects to forecast growth in iPod sales - which have been exponential. We conducted a similar study years ago, but used made up data, we thought it would be better to use a real exponential data series, so we re-did the study this time using iPod sales as the data series to forecast. Now, it turns out humans are poor at forecasting exponential growth - there is a cognitive bias at work related to the anchoring and adjustment heuristic - which means we just don't pick up on the exponential nature of a data series and forecast growth as a straight line and as a result under estimate growth of exponential data.

In our experiment, we gave the subjects some historical quarterly data, and asked them to forecast 2 quarters out (we knew that "actuals" for the periods we were asking them to forecast).

The idea we designed the experiment to test is a simple one. If you take a log of exponential data, you get a straight line. Humans are good at doing straight line forecasting so we reasoned that if you take a log of exponential data, forecast based on that, you'll get a better forecast than if you just have the data in its 'raw' state. The conversion to log data and back is something a computer system - a DSS - can do nicely, so that's the basic shape of the experiment. All the detail is in the paper - as you'd expect there is a control group using a paper based version of the data, but we built a nice little tool to perform the forecast. You can click on a chart to make a forecast - and it shows you the number, or type in a number and it shows where that number is on the chart. One version of the tool had just the raw data, the other showed both the raw data and the log data.

So, to the results ... the computer supported forecasts were better. Phew, often in these types of studies the DSS is of no help. In our case it was. However, the simpler version of the system, did better than the version that had the log data - the opposite of what we expected. Our explanation is that the simple version encouraged experimentation, letting the users think a bit more about their forecast - exactly what you want a DSS to to. However, rather than helping, the more complex system with the log data, intimidated the users, stopping them from experimenting and as a result they made poorer forecasts.

So forget about forecasting and exponential growth, the main lesson from this study is keep the interface simple.

POD

A survey of Australian BI practice - call for participation

2008-11-21T20:58:00.002+11:00

We really like the folks at Intalign and are doing a couple of research related activities together with them on an ad hoc basis. Right now they are doing a survey of Australian organisations to understand what they are doing with BI. Martin Kratky, one of the Directors at Intalign, just sent out this update and reminder to people on his list about the survey:

"So far, more than 50 major Australian companies have participated in this important study, held in conjunction with Monash University. To gain the broadest possible cross section of prominent Australian organisations, we would be very appreciative of your confidential input.

As a thank you for participating in this survey (max. 10 mins), we would like to offer you:

A free copy of the survey results to benchmark your BI processes including reporting, software, budget allocations and more against best practices and other survey participant's organisations.

A $150 voucher to be used for a one day executive workshop

We are still inviting professionals that are in charge of Business Intelligence processes in their organisation (with an annual turnover of at least $20 million) to take part in the survey. To participate, please send an email to: survey@intalign.com."

If you fit the demographic - an Australian organisation with turnover > $20 mil, and have some spare 10-15 minutes, why not drop them an e-mail and get the link to the survey.

Peter

The vendors ARE mad!

2008-11-14T00:16:00.003+11:00

Now I need to qualify this post - which is in response to a vendor presentation I went to during the week - by saying, that the vendor - Cognos - is in my opinion one of the good ones. Their technology is pretty good, maybe still a few performance issues with large numbers of users but generally in great shape. I really like the new report meta-data feature (lineage they call it) in version 8.4 of their core reporting tool. That will be quickly copied by the other vendors.

But, and I mean BUT, I saw a demo of a guided report writing tool - intended to help end-users build their own reports without IT support. All the vendors offer something like this, and none of them work very well.

I counted clicks during the demonstartion. The presenter went through a standard demo, just building a simple financial report with revenue, target and variance shown. It took 31 clicks and 7 drag and drop actions to the build the report. The presenter didn't make a single mistake, knew the software backwards and knew where the data was, and it still took 31 clicks and 7 drop and drags. NO. That is not and end-user report creation tool.

Arrrgggghhhhh.

POD
P.S. They also had 3D pie charts with the middle taken out to make them look like donuts. Why?

The iPhone

2008-10-28T20:18:00.002+11:00

I have an iPhone and I love it. I hate phones, and I hate mobile phones in particular .. but my iPhone is different. Really. the reaction I get to it, when people see it reminds me of the "old" days when Windows was young and DOS ruled and Macs' were the only platform that really implemented a WIMP interface (only when Windows became better accepted - really after the release of Windows95 did mainstream people start calling the bit-mapped, mouse driven graphic desktop-metaphor based interface a GUI).

DOS folks (and I was one once too!) would laugh at me and complain that I was letting the computer make decision for me, I was missing out on the power of commands like "Copy *.*" - look at all the clicks you have to do to copy all your files, ... and on it would go.

Many early EIS systems were Mac-based. To write for DOS mean't the vendor's had to create their own GUI - a non-trivial exercise - so the Mac was a preferred platform. (In 1990 we evaluated Pilot, Comshare and Holos - all had Mac versions that were heaps better than the DOS versions.

Along came Windows 3.11 and then Windows 95 and the game changed.

The Desktop metaphor was created at Xerox Parc in the 60s. It is still with us. In a less defined and less consistent way our interfaces are now also dominated by a document metaphor (partly due to the take up of the Web).

The reason why the iPhone is so cool - and so interesting to me - is not that it's a phone, or that is has a touch screen, or GPS, or a camera or whatever other feature it has - all of those are neat, but as my Windows Mobile/Blackberry/Palm/HTC loving friends point out there are other phones with similar or better technical feature sets. The thing is its easy to use - and it not because it has a great screen, and a touch interface. The interface is based on a "new" metaphor. Information is displayed on a surface - the information becomes the interface. That's the revolution. Many of my friends don't get that.

Sadly, at this stage, it seems very few BI vendors get it either. This metaphor - and the technology that supports it - is perfect for BI systems. Watch this video from Edward Tufte - he's talking about the iPhone as a phone - forget that and just think about what he says about the interface. He's describing an almost perfect way of presenting and exploring data. As he explains it, the iPhone is a dramatic new paradigm. It will be copied and probably improved upon - but not by phone vendors who think squeezing a menu based system onto a touch screen is an appropriate interface for a mobile device.

I hope that one day the BI vendors will understand too. I wonder - if as before - we have to wait for Microsoft (who are playing with this new metaphor with their surface product) to build a new version of Windows before the BI worlds makes a real change for the better.

Reblog: Godin Causes a Data Visualisation Storm

2008-07-16T09:27:00.004+10:00

A quick post with no content of my own:

Seth Godin, professional agent provocateur, has caused something of an outcry amongst data visualisation aficionados in a blog post lambasting the use of bar charts in presentations. Although his heart is in the right place, Godin's somewhat simplistic prescriptions for improving communication effectiveness miss the mark. Perhaps his greatest crime is to advocate the use of pie charts, which, as we know, are as "professional as a pair of assless chaps." The boys at Juice Analytics and Stephen Few have the scoop.

IT Folks are Luddites

2008-05-30T11:59:00.003+10:00

A while ago in a previous post, I referred to an interview with Neil Raden entitled "Is Business Intelligence Stuck in the Past?" where he talked about how most business users today are more tech-savvy than the IT department. Peter O'Donnell just sent me through a link to a Gartner Voice interview with Peter Keen (one of the founding fathers of DSS/BI) where he makes essentially the same point:

IT is in danger of becoming the technology laggard.

Tune in here to listen to the interview. You can also subscribe to the Gartner Voice podcast in iTunes (the interview with Keen is the most recent episode).

Data Warehouse / BI Security

2008-04-17T12:27:00.005+10:00

Peter O'Donnell and myself are currently supervising an honours student who is looking at the issue of data warehouse security, with a view to doing a survey of DW security practices in Australian companies. It's still early days, but one of the things that Justin has found is that there is very little literature (academic or otherwise) talking about the issue (either highlighting problems, or outlining best practice). This is both good and bad news: it means that Justin will be making a real contribution, but he's going to have trouble writing the literature review section of his thesis!

To give you some idea of where our thinking is at, here's a generic architecture for the flow of information through a data warehouse:

Each component of the diagram above is a potential security problem. Just the ETL process, for example, poses problems of massive amounts of data moving around a network, taken out of what is presumably an initially secure environment. We've found very little that talks about securing the individual components of the architecture, or of taking an holistic view and securing the whole process, end-to-end. On the flip-side, security often poses a problem from a functionality or performance perspective - what can we do to make the whole thing as responsive and functional as possible while still protecting an important organisational assett?

Any thoughts, war stories, pointers to resources or comments would be appreciated!

DSS Governance Paper

2008-04-01T16:18:00.002+11:00

Here's a link to the final paper that I mentioned previously in the post on DSS governance. Hopefully some people find it useful and/or thought provoking.

Trends in Data Warehousing

2008-03-19T14:52:00.013+11:00

Last year I co-authored a book chapter with two other colleagues, Peter O'Donnell and David Arnott, on the use of data warehouses for decision support, and it's just recently been published. The book is called Handbook on Decision Support Systems edited by Frada Burstein (another Monash colleague) and Clyde Holsapple. One section of the chapter that I wrote looked at current trends in DW practice, and I thought, as I wrote it in late 2006, that it would probably be better as a blog post, than part of a chapter in a (hopefully long-lived) book. Here's the excerpt. I'd be interested to hear what other people think are the big trends in DW and where it's headed.

Current Trends and the Future of Data Warehousing Practice

Forecasting future trends in any area of technology is always an exercise in inaccuracy, but there are a number of noticeable trends which will have a significant impact in the short-to-medium term. Many of these are a result of improvements and innovations in the underlying hardware and database management system (DBMS) software. The most obvious of these is the steady increase in the size and speed of data warehouses connected to the steady increase in processing power of CPUs available today, improvements in parallel processing technologies for databases, and decreasing prices for data storage. This trend can be seen in the results of Winter Corporation's "Top Ten Program," which surveys companies and reports on the top ten transaction-processing and data warehouse databases, according to several different measures. Figure 11 depicts the increase in reported data warehouse sizes from the 2003 and 2005 surveys (2007 data has not yet been released):

Ten Largest Global Data Warehouses by Database Size, 2003/2005. From Winter Corporation.

The data warehousing industry has seen a number of recent changes that will continue to have an impact on data warehouse deployments in the short-to-medium term. One of these is the introduction by several vendors, such as Teradata, Netezza and DATAllegro, of the concept of a data warehouse 'appliance' (Russom, 2005). The idea of an appliance is a scalable, plug-and-play combination of hardware and DBMS that an organization can purchase and deploy with minimal configuration. The concept is not uncontroversial (see Gaskell, 2005 for instance), but is marketed heavily by some vendors never-the-less.

Another controversial current trend is the concept of 'active' data warehousing. Traditionally, the refresh of data in a data warehouse occurs at regular, fixed points of time in a batch-mode. This means that data in the data warehouse is always out of date by a small amount of time (since the last execution of the ETL process). Active data warehousing is an attempt to approach real-time, constant refreshing of the data in the warehouse: as transactions are processed in source systems, new data flows through immediately to the warehouse. To date, however, there has been very limited success in achieving this, as it depends on not just the warehouse itself, but performance and load on source systems to be able to handle the increased data handling. Many ETL processes are scheduled to execute at times of minimal load (eg. overnight or on weekends), but active warehousing shifts this processing to peak times for transaction-processing systems. Added to this are the minimal benefits that can be derived from having up-to-the-second data in the data warehouse, with most uses of the data not so time-sensitive that decisions made would be any different. As a result, the rhetoric of active data warehousing has shifted to "right-time" data warehousing (see Linstedt, 2006 for instance), which relaxes the real-time requirement for a more achievable 'data when it's needed' standard. How this right-time approach differs significantly in practice from standard scheduling of ETL processing is unclear.

Other than issues of hardware and software, a number of governance issues are introducing change to the industry. One of these is the prevalence of outsourcing information systems - in particular the transaction-processing systems that provide the source data for warehouse projects. With many of these systems operated by third party vendors, governed by service level agreements that do not cover extraction of data for warehouses, data warehouse developers are facing greater difficulties in getting access to source systems. Arnott (2006) describes one such project where the client organization had no IT staff at all, and all 13 source systems were operated off-site. The outsourcing issue is compounded by data quality problems, which is a common occurrence. Resolution of data quality problems is difficult even when source systems are operated in-house: political confrontations over who should pay for rectifying data quality problems, and even recognition of data quality as a problem (in many cases, it's only a problem for data warehouse developers, as the transaction processing system that provides the source data is able to cope with the prevailing level of data quality) can be difficult to overcome. When the system is operated off-site and in accordance with a contractual service level agreement that may not have anticipated the development of a data warehouse, they become even more difficult to resolve.

In addition to the issues of outsourcing, alternative software development and licensing approaches are becoming more commonplace. In particular, a number of open source vendors have released data warehousing products, such as Greenplum's Bizgres DBMS (also sold as an appliance) based on the Postgres relational DBMS. Other open source tools such as MySQL have also been used as the platform for data warehousing projects (Ashenfelter, 2006). The benefits of the open source model are not predominantly to do with the licensing costs (the most obvious difference to proprietary licensing models), but rather have more to do with increased flexibility, freedom from a relentless upgrade cycle, and varied support resources that are not deprecated when a new version of the software is released (Wheatley, 2004). Hand-in-hand with alternative licensing models is the use of new approaches to software development, such as Agile methodologies (see http://www.agilealliance.org) (Ashenfelter, 2006). The adaptive, prototyping oriented approaches of the Agile methods are probably well suited to the adaptive and changing requirements that drive data warehouse development.

The increased use of enterprise resource planning (ERP) systems is also having an impact on the data warehousing industry at present. Although ERP systems have quite different design requirements to data warehouses, vendors such as SAP are producing add-on modules (SAP Business Warehouse) that aim to provide business intelligence-style reporting and analysis services without the need for a separate data warehouse. The reasoning behind such systems is obvious: since an ERP system is an integrated tool capturing transaction data in a single location, the database resembles a data warehouse, insofar as it's a centralized, integrated repository. However, the design aims of a data warehouse that dictate the radically different approach to data design described above in Sections 3.1 and 4 mean that adequate support for management decision-making requires something other than simply adding a reporting module to an ERP system. Regardless, the increased usage of ERP systems means that data warehouses will need to interface with these tools more and more. This will further drive the market for employees with the requisite skill set to work with the underlying data models and databases driving common ERP systems.

Finally, Microsoft's continued development of their Microsoft SQL Server database engine has produced a major impact on Business Intelligence vendors. Because of Microsoft's domination of end-user's desktops, it is able to integrate its BI tools with other productivity applications such as Microsoft Excel, Microsoft Word and Microsoft PowerPoint with more ease than their competitors. The dominance of Microsoft on the desktop, combined with the pricing of SQL Server, and the bundling of BI tools with the DBMS means that many business users already have significant BI infrastructure available to them, without purchasing expensive software from other BI vendors. Although SQL Server has been traditionally regarded as a mid-range DBMS, not suitable for large-scale data warehouses, Microsoft is actively battling this perception. They recently announced a project to develop very large data warehouse applications for an external and an internal client, to handle data volumes up to 270 terabytes (Computerworld, 2006). If Microsoft are able to dispel the perception that SQL Server is only suited for mid-scale applications, it will put them into direct competition with large-scale vendors such as Oracle, IBM and Teradata, with significantly lower license fees. Even if this is not achieved, the effect that Microsoft has had on business intelligence vendors will flow through to data warehousing vendors, with many changes being driven by perceptions of what Microsoft will be doing with forthcoming product releases.

Threat Level "Burgundy, If You Will"

2008-02-29T16:27:00.003+11:00

Just when you thought you'd seen every stupid data visualisation trick out there, someone invents the "magic pie chart" and the rotating "statistical lazy susan." The US cable news networks are outdoing themselves during the US presidential primaries, and breaking just about every rule in the data visualisation book. BI vendors, eat your hearts out! Check out this gem from The Daily Show with Jon Stewart:

A Response to DSS Governance

2007-12-12T12:07:00.001+11:00

Every couple of years or so, you get one of those rare students in a class - one who actually teaches you something about the topic. Bruce Fowler was one of those students in a Data Warehousing course that I taught earlier this year. He's a data warehouse manager for a resource management company, and so has an interest in DW management and governance issues.

He posted a response to the DSS Governance post I put up recently, but due to the new comments system that Peter's been playing around with, it didn't come through. It was substantive enough, and included some diagrams, that I thought it reasonable to post (with Bruce's permission!) his comments as a separate entry. All images in (and linked to by) this entry are Copyright © 2007 Bruce Fowler

The comparison of IT, railroads and power as "infrastructure technologies" - sharing characteristics of competitive advantage, ubiquity and finally commoditisation (followed by a loss of strategic benefit and value) is a fairly long bow. It is important to remember that the catalyst for the demise of rail was not its cost or availability (at least not directly), it was the advent of more cost effective and time efficient technological alternatives (combustion engines, aeroplanes). I am not sure I would support the contention that the commoditisation of a technology has any specific correlation to the technology’s loss of strategic initiative or competitive advantage.
Railway lines continue to provide cost effective and logistically efficient means of transporting large volumes of material across the country (provided the infrastructure exists and the alternative means remain less cost effective and time efficient), and are being used in new and innovative ways to supplement conventional income streams for logistics organisations through integrated fibre networks. Power companies continue to explore delivery of new products and services over existing infrastructure (i.e. broadband over powerlines), and are currently reinventing themselves in biofuels space to enable delivery of “green” energy to a more environmentally conscious market. Combustion engines are being redesigned to be more fuel efficient and “environmentally friendly”.
The delivery platforms have been around for some time and form part of our everyday lives – their use is evolving in new and innovative ways. IT – perhaps more than any other technology platform – has the capacity to continue to be adapted and evolved to meet the ever-changing demands of its user base. Commodity? Yes. Does it matter? Of course it does.
Back on topic …
I suspect there is a significant difference in the structure, objectives and necessities of Corporate Governance and IT Governance; and of the relationship between the two in comparison to the same for DW/DSS Governance. Even then, there are perhaps different factors that need to be considered from the perspective of DW Governance versus DSS Governance, and their respective relationships with IT Governance.
Consider the basis for the introduction of Corporate Governance – a means of managing the seemingly inevitable consequences of the centralisation of power and decision making authority (Husted, 1999); then consider the basis for the introduction of IT Governance – the patterns of authority for the significant IT activities of an organisation including IT Infrastructure, IT Use and IT Project Management. In simplistic terms, the former focuses on risk mitigation, the latter on efficiency and effectiveness.
Over time, the use of Corporate Governance models as a direct risk mitigation strategy has given way to an army of formalised standards, auditing and reporting obligations – spanning multiple levels of business, including technology operations – administered by committees through levels of delegation of authority. The line between Corporate Governance and IT Governance has blurred, and the need to maintain alignment of IT (infrastructure, use and project management) with the organisations mission objectives is now forefront in the minds of most informed corporate executives. This alignment recognises that the critical strategic importance of IT to successful business operation.
Get it right, and IT can (at a minimum) provide a stable platform from which other strategic endeavours can be launched. Get it wrong, and a failed IT system or project can (in the best of cases) reduce your business efficiency or effectiveness, or (in the most severe of circumstances) end your business (someone say ERP?).
A pencil is a commodity. IT is a tool that can be used to create competitive business advantage, or as easily be misused resulting in catastrophic business or process failure.
I find myself off topic again …
The evolution of DW Governance arrangements depends on the confluence of many factors that interact with one another in a number of complex ways (Sambamurthy and Zmud, 1999). The key factors, their interactions and dependencies are included below.
Given the nature of the relationships identified below, perhaps we could identify the interactions has a loosely coupled hierarchy: with each child exhibiting some characteristics of its parent, and the will and initiative to move around (and sometimes break out of) the boundaries defined by the ever-watchful parent (who will evolve and adapt their boundaries to meet the growing needs and demands of the child, but have the foresight and capacity to bring the child back into line if needs be).

Thanks Bruce. Generally, I vigorously agree with everything you've written here. If you would like a copy of Bruce's original diagrams, drop me a line and I'll pass on the request.

DSS Governance

2007-12-10T11:17:00.000+11:00

In the last post, I mentioned I had been looking at the issue of governance and DSS. In fact, this is something I've been thinking about since a student asked in a lecture if there was anything on data warehouse governance a couple of years ago, and I've just written a paper for the bi-annual conference for the academic DSS community, IFIP Working Group 8.3.

The paper is currently under review, so I won't post it here yet (I'll put up a link when it's gone through that process), but I thought I'd put the basic argument out there for people to comment on, since it's all still conceptual at this stage.

IT governance is an important topic - on the one hand corporate governance is a big thing; and on the other, we've got Nicholas Carr telling us that IT is not a strategic advantage for organisations. The IT industry needs to ensure that we're managing an important corporate resource effectively.

As a significant chunk of the IT industry DSS (ie. BI) is all a part of IT governance. Unfortunately, there's not a lot of academic work that talks about how to do this effectively for DSS (there's a bit on data warehousing, but that's it).

The argument I make in the paper is based on the idea that DSS is different to other kinds of IT in two ways:

DSS are chaotic systems. They evolve. They can and should evolve quickly. If they don't evolve, then there's something wrong: learning isn't taking place, and the system isn't doing what it's supposed to: provide support with semi- or un-structured decision-making. Sure, evolutionary development is used to build all kinds of systems, but there is usually some end-point in mind where the system becomes stable (relatively). This isn't the case for DSS.
DSS are subversive systems. They're designed to deal with strategic decisions (a corollary of being built for semi- and un-structured decisions). Their use deliberately changes some aspect of an organisation's structure (not the physical structure, but the organisation's strategic direction, policies, values, procedures, work-flows, etc.). Other systems may have this effect too, but often it's not deliberate. With DSS it's intentional - part of it's raison d'etre.

IT governance is largely about how to control IT resources, enforce standards, and manage changes in a methodical fashion. It's based on a mindset of stability and prediction. Although there's been a lot written on IT governance (check out Weill & Ross's excellent book on IT governance), and a lot of it focuses on the appropriate approach for given organisational types (eg. centralised versus decentralised management cultures), there's nothing I've seen that actually takes characteristics of the technology into account. My assertion in the paper is that the underlying assumptions of a given governance approach should be consistent with the underlying assumptions embedded in the technology being governed.

Bureaucratic approaches that are appropriate for managing technologies like transaction-processing systems - steering committees, IT councils, service level agreements, etc - are inappropriate for chaotic systems. Enforcement of an organisational structure on the operation of a technology is inappropriate for a technology designed to question and change that same structure. Excessive control can (and has, we've seen it) stifle and eventually kill a DSS project.

The conclusion that I come to is that for DSS to thrive, the developers and users need the autonomy to play around with the system's design and functionality without going through multiple layers of bureaucracy. DSS should operate, therefore, in a kind of 'governance sandbox', where the DSS team are trusted to do the right thing as they see it. This kind of approach needs some clear boundaries however, including clear goals and objectives, and what constitutes overstepping the mark. This in turn requires a pre-existing, well planned general IT governance strategy.

Anyway, those are my current thoughts. Feel free to shoot through any comments. A couple of issues spring to mind, such as how do different kinds of DSS technologies differ in their governance requirements - eg. data warehouses versus dashboards versus small-scale throwaway spreadsheets. What specific mechanisms work for DSS governance inside the 'sandbox'? How much scope should DSS developers and users have? All food for future research...

IT Archaeology: Whatever Happened to SDS?

2007-11-12T10:39:00.000+11:00

I've been digging through some ancient texts (in IT terms) over the past week or so, looking at the issue of IT governance and how it relates to the development of decision support systems. In doing so, I read again an article from 1971 in Sloan Management Review that first coined the term 'decision support system' written by two academics from MIT: Anthony Gorry and Michael Scott Morton.

In defining DSS, they also designed a second class of information system that I'd completely forgotten about, known as a 'structured decision system' (SDS). The term 'structured' comes from an adaptation of a model of decision-types by Nobel laureate Herbert Simon. In Gorry & Scott Morton's terms, decisions are either structured (well-understood, fairly easy to resolve, lend themselves to well-defined workflows or decision-rules), unstructured (difficult, high levels of ambiguity, no clear process for making the decision) or somewhere in-between (semi-structured).

Gorry & Scott Morton argued that such systems are designed to support semi- and un-structured decision problems. For structured decisions like inventory control, short-term forecasting and so on, they coined the term SDS.

This got me thinking. I had always seen today's BI systems as the inheritor of the DSS concept - basically the latest term in a long string of marketing names for systems designed to support managerial decision making. Now, I'm not so sure. In looking at Gorry & Scott Morton's definitions, most BI tools seem to be targetted more at structured, rather than unstructured decisions. Couple this with efforts by people like Howard Dresner to shift the BI concept more and more to enterprise reporting, and I reckon that BI is more about SDS than DSS.

Which is a problem.

There is a class of decisions that really need the support of systems that can help the decision-maker through the decision-making process by embedding principles of good decision-making in the system, and helping them make sense of the information they have. These decisions tend to be strategic and important, and the potential ROI for a system that can improve decision-making in this area goes way beyond being able to run an operational report that used to take hours in 30 seconds. In other words, there is a real need for the decision support systems that Gorry & Scott Morton described 36 years ago, and I don't think that BI tools are currently being used to build them.

This is not to say that they're not being built, of course. Instead, I think they tend to fly under the radar more, as individual strategic decision-makers throw together systems to answer specific questions in a way that they're comfortable with: witness the continued pervasion of Excel in the upper echelons of organisations despite the flash-wizardry of the latest 3D-pie-charting engine; skunk-works projects for individual managers; the explosion of independent data marts.

As I said above, the impact of the effective use of a DSS far outstrips that of a SDS because the decisions made fundamentally and directly affect the strategic direction of a firm. It's critical that good quality DSS are developed, and I don't think we're going about it in the right way at the moment. Excel can be dangerous if not used properly, but because everyone is focused on the highly-visible SDS-like BI projects, the DSS-needs of an organisation are often addressed in an ad hoc way.

It's a pity that we don't pay attention more to what was written in the past about IT. Not only do you realise that we keep making the same mistakes despite the changing technology, you come across interesting ideas that can change the way you perceive today's industry.

The Myth of BI for the Masses

2007-10-24T16:30:00.000+10:00

Glenn Alsup over at the Viewmark Blog has a great post on how it's next to impossible to properly satisfy all user needs with a single analytics tool/data warehouse. I've blogged on this theme before, but it's worth hammering home time and again, since it's diametrically opposed to what the vendors want you to think. The problem with BI is knowing the information requirements prior to users using the system. Given the role of BI is to support decision making, people don't know what they need until they start to work through the issues - the very thing the BI system is supposed to support. No data warehouse design, however big, is ever going to be able to anticipate the specific information requirements needed by a decision-maker, especially if the decision is a strategic one (the real sweet-spot for BI).

My personal view is that it's better to spend money on a team with some great analytical skills than a software solution that ends up as a glorified enterprise reporting tool. In many decision situations, small-scale, non-permanent 'ephemeral' systems (so-called 'personal DSS') thrown together to answer specific questions are more useful than multi-million dollar BI systems. But then that doesn't sell software licenses, consulting or support contracts.

BI Marketers are eevil. As in fruuuuits of the deveeeil.

2007-10-24T16:04:00.000+10:00

Just received via spam from a well known vendor:

Markets Demand Accurate Forecasts - Learn how to improve your forecasts by a minimum 50%

If I'm likely to fall for a line like that, I'd be better off doing a basic stats course to improve my forecasting accuracy. It's silly statements like this that give BI vendors the reputation they have today. Shame on you.

CRM: Seeing things from other people's perspectives.

2007-10-23T15:00:00.000+10:00

We've just finished teaching for the semester here at Monash, and one of the subjects I taught was a Masters level unit on customer relationship management. As part of teaching the unit, I created a blog, which I've just switched off, but thought this post was worth saving, and relevant to this blog here. It was originally posted on the 2nd of August, 2007, and appears slightly edited (for context) here.

Sometimes we lose sight of the point of our business initiatives, failing to put ourselves in the shoes of stakeholders like customers (or users, in the case of BI).

Fournier, S., Dobscha, S. & Mick, D.G. (1998) Preventing the Premature Death of Relationship Marketing, Harvard Business Review, Jan/Feb98, V. 76, I. 1, pp. 42-51

I came across the HBR article above today and was struck by its relevance to the Barry Schwartz video below* (and the one viewed in this week's seminar). Although it's a bit old now (nearly 10 years), the article talks about how the idea of relationship marketing (the underpinning of CRM) is often subverted by the very activities marketers engage in to fulfil it. The relevance of Schwartz's idea of the paradox of choice to CRM is that it raises doubts about this ideal of the one-to-one relationship between a customer and a company. The article above builds on this theme very nicely. From the article:

Every company wants the rewards of long-term, committed partnerships. But people maintain literally hundreds of one-to-one relationships in their personal lives - with spouses, co-workers, casual acquaintances. And clearly, only a hadnful of them are of a close and committed nature. How can we expect people to do anymore in their lives as consumers?

"It's overkill," said one woman we interviewed, referencing the number of advances from companies wanting to initiate or improve their relationship with her. "One is more meaningless than the next."

The article points out that consumer-satisfaction is at an all-time low, despite all these relationship-marketing efforts. I reckon a lot of that has to do with the phenomenon that Schwartz talks about, but relationship-marketing, aided and abetted by CRM, seems to only exacerbate it all. You have to wonder how effective our BI systems are at doing what they try to do: improve the decision-making process.

[* The Schwartz video referred to was posted on the original blog - it is a presentation by Barry Schwartz to Google on his concept of the Paradox of Choice. You can watch it here. It's worth watching in it's own right. ]

10 Keys to BI Success

2007-10-23T12:57:00.000+10:00

This CIO magazine headline (10 Keys to a Successful Business Intelligence Strategy) popped up in the links to the right recently, and it's worth the read. It's spot on about most things, especially recommendations nine and ten that you should start simple, go for 'low-hanging fruit'. Author Diann Daniel is also quite right in pointing out that the main driver needs to be a c-level executive other than the CIO.

I'm not a fan of the 'X keys/steps to "insert good thing here"' approach to journalism, but good stuff nevertheless.

SAP & Business Objects

2007-10-11T09:59:00.000+10:00

Everyone will have heard by now about the massive buyout of Business Objects by SAP. At just under US$7 billion, this is almost twice as much as Oracle paid for Hyperion several months ago (ok, 7 billion US is not that much anymore, but 4.7 billion euro is a nice stash of cash). While the deal is still awaiting approval by shareholders and regulators, it's probable it will go ahead.

The most obvious observation to be made about the deal is that it's a reaction to the Oracle/Hyperion deal, which it no doubt is, in part. There is another angle, though, touched on by Joshua Greenbaum over at Enterprise Anti-matter. One of the points he makes is that the ERP market is starting to stagnate as a result of market saturation. This is certainly the case in the market here in Australia: a conversation I had with a contact in a large BI/Data Warehousing company highlighted the fact that a lot of former ERP consultants are applying for jobs in business intelligence. If the industry research companies are to be believed, BI is still experiencing growth and is apparently on every CIO's 'radar' for the next year (I am so sick of military metaphors for business...).

Joshua also mentions the different perspectives that ERP and BI companies have on what it is that they do, with ERP obviously focussed on transactions, while BI is tools-oriented. This is certainly true, but I think that it goes further than that.

Transaction-oriented systems like ERP or any OLTP system, have inherently different design requirements from systems designed to support decision-making. Transaction systems need to process lots of small packets of data very quickly, are designed for efficient data entry and storage, and are mainly used by people with relatively little organisational authority. They are also, comparatively, easy to design. The workflows and processes supported by these systems (or in the case of ERP, imposed by these systems) are well understood and possible to document in minute detail.

Decision support tools, though, are designed to answer questions and facilitate a decision-making process. Making decisions is a fundamentally different kind of activity to keeping track of business events. The need for a decision-support tool necessarily means that the users (ie. decision-makers) are functioning in an environment characterised by uncertainty. They may have an idea of the kinds of questions they need to answer, but they can't be absolutely sure that this list is complete (and practice shows that it never is). Indeed, in answering those questions, a whole range of new questions invariably arises. In short, the use of a decision-support tool is fundamentally a learning process:

The users don't know exactly what they need
They don't necessarily need what they want
Use of the system itself changes their understanding of what they need
The system therefore needs to change to help answer new questions (and back to 3 again..)

Learning is an evolutionary process, which means the development and use of decision-support tools also needs to be evolutionary. The difference between ERP and BI is not just a tools-based perspective, it's a need for a fundamentally different mindset on the development and use of the system. When you throw users with a massive amount of organisational clout (senior executives) and cognitive factors associated with decision-making into the mix, the practice of BI and ERP are worlds apart.

Unfortunately, a lot of IT folk don't understand this difference, and we see time and again the use of old-fashioned engineering approaches to BI projects. Such approaches may work fine for transactional systems, like ERP, but doom many BI projects to failure. SAP's own BI module, SAP/BW is widely loathed as a decision-support tool, in large part because it was developed by people with a transactional-mind set. I'm aware of one very large BI project that is currently underway (not using Business Objects or SAP/BW) that exhibits exactly this engineering approach: 18 months into the project, not a single user-facing aspect of the system has been delivered to anyone, with all the work going to the back-end infrastructure. Without something for the users to react against, the learning process can't kick-off, and the requirements (against which all this infrastructure is being designed) cannot possibly be properly understood. Down the track, either one of two things will happen (unless something changes): either the requirements will change and the system will have to be redesigned; or requirements will change, the system won't be updated, and no-one will use it because it doesn't answer their questions. The only way around this is to deliver some quick wins in the form of one or more pilot projects that focus on easy, yet strategically important business areas - get the system in front of some user eyeballs, and the requirements elicitation process will be driven by their feedback.

In any case, to bring this back to SAP and Business Objects, the move makes a lot of sense for SAP. They pick up expertise on developing systems for decision support, not just recording transactions. Hopefully, they recognise this, and the move is not just motivated by a reaction to Oracle and Hyperion, or to tap into a growth market. The fact that SAP are saying that Business Objects will continue to operate as a separate business suggests they want to keep the skill base around, and hopefully it will translate into some skills transfer in the direction of SAP.