June 19, 4:00 EST

The promise of Hadoop can be seen in all kinds of ways -- the proliferation of open source projects; the virtually limitless applications of Big Data; the sheer number of vendors getting involved. But the real value only comes from a mature environment, and that's Hadoop 2.0. What are the component parts of a robust solution? How are today's cutting-edge organizations leveraging the power of Big Data?

Register for this episode of Hot Technologies to hear veteran Analysts Dr. Robin Bloor of The Bloor Group, and Rick Sherman of Athena IT Solutions, as they offer perspective on how the Hadoop movement is shaping up. Larry Weber of IBM will then offer his take on the tools and architecture necessary to tackle the new challenges posed by Big Data. He'll discuss IBM's latest big data offerings including IBM InfoSphere BigInsights, IBM InfoSphere Streams, and IBM InfoSphere Data Explorer, and IBM's vision for simplifying an organization's big data journey.

Register:  https://bloorgroup.webex.com/bloorgroup/onstage/g.php?t=a&d=667671123

We've all heard the selling points of cloud computing: low initial cost, ease of deployment, lack of maintenance concerns. But what about the downsides? In the long run, the costs can start to rival those of on-premise solutions, and coming back down from the cloud can prove somewhat tricky. And then there's loss of control. Indeed, the concept of “hybrid” cloud solutions will likely prevail for sometime. And there are some things that should stay safely on the ground.

Register for this episode of DM Radio to hear hosts Eric Kavanagh and Justin Kern interview John O'Brien of Radiant Advisors, plus Peter Zaitsev of Percona, Kelly Battles of Host Analytics and Rick Sherman of Athena IT Solutions.

Getting exactly what you want is a principle that was most deliciously embodied by Steve’s Ice Cream, an institution beloved by Boston-area college students beginning in 1973. Steve pioneered the “smoosh-in,” adding customer-selected goodies like crushed Heath Bars® and M&Ms® to his rich, slow-churned ice cream. (Now you can buy ice cream like this in any grocery store, but in the 1970s you had to wait in line at Steve’s Somerville store.)

Like the original Steve’s ice cream, BI is best when you pick and choose.  

So why do Enterprise IT groups often standardize on the equivalent of a pint of vanilla when selecting hardware and software products? There is a variety of reasons:

• Lower costs -- there is only one tool to learn, use and support
• More expertise -- they can develop more expertise by concentrating on one tool
• Potential lowered licensing costs
• Better vendor support --  “one neck to choke” (a phrase that I hate)

When choosing a BI solution, the IT group typically goes through a software evaluation or “bake-off,” selecting tools based on who scores the most across a multitude of criteria. The winner is typically a full-stack BI vendor because it has the most extensive feature checklist and when you add up the scores it has the highest number. In addition, enterprise IT groups will use the ratings from industry research analysts to justify their selection. (The analysts themselves likely went through a feature checklist selecting the most feature-rich product.)

An enterprise’s business community will have a very diverse set of analytical needs and work styles. The type of analysis they perform will vary based on the depth, subject, volume, and structure of the data used, as well as the business processes: examining salespeople’s performance, providing customer support, predicting customer behavior, etc. Business people will also approach analysis differently based on their background and experience.

Although BI tools have been around for a couple of decades, the depth, breadth and variety of BI capabilities and styles continues to expand and change. A decade ago, dashboards and scorecards were the rage as compared with the typical reporting tools at that time. This was followed by On-Line Analytical Processing (OLAP) and ad-hoc query tools.  BI options now include data discovery, data visualization, in-memory analytics, predictive modeling, BI appliances and Big Data analytics.

The BI products that enterprises standardize on provide solid BI functionality, but even the full-stack BI vendors do not provide all the BI capabilities and analytical styles that business people could use. The reality is that much of the analytical innovation comes from emerging or smaller vendors. The full-stack vendors may build comparable capabilities or acquire these innovative products after they see them being adopted, but it may be years for that to happen.  Meanwhile, the enterprises that wait may be placing their business community in a competitive disadvantage.

The best approach that IT can take is to first determine the variety of BI capabilities and styles that their business community needs, and then expand their BI portfolio rather than trying to channel everyone into a one-size-fits-all approach. This process should not to encourage a free-for-all with BI tools purchased just for variety sake, but rather focus on filling in legitimate gaps in what their BI standard tool does not support today.

As anyone who waited in line for a serving of Steve’s famous ice cream with M&Ms and crushed Reece’s Peanut Butter Cups® will tell you, selecting the perfect combination for your needs is the only way to go.

Self-Service BI Truths

The primary goal of self-service BI is to enable business people to perform business analytics in a timely and self-sufficient manner. A secondary goal for many business people is that the information they use in that analysis is consistent, comprehensive, correct and current.

Supporting objectives:

• More pervasive and in-depth use of analytics in decision-making
• Reduced time to insight
• Improved business and IT productivity

Self-Service BI Myths

There are too many myths and hopes that impede successful self-service BI efforts:

• Business people want to build their own reports. No they don’t, but they do want to perform their own analysis.
• You need to eliminate IT from BI. No, it’s not that simple. It’s naïve to think that you can give business people the latest wave of BI tools and avoid working with IT. Although an enterprise does want to get IT staff out of the perpetual queue of creating BI reports and dashboards, or writing custom ad-hoc queries to support new or modified business analytical requirements, business people will get frustrated without IT involvement (we’ll discuss in what areas later)  and likely create the next wave of data shadow systems that are data silos.

• All you need is the latest self-service BI tool. Not even close. This is a great sales pitch that both business and IT would love to be true. It would great if buying a BI tool eliminated the hard work of, for example, getting the data consistent and governed, but unfortunately if it was just technology then self-service BI would have been the norm years ago.
• The problem is that business people just cannot access the data they want. No, the problem is not access, it’s the data inconsistency and quality. This is the sales pitch in almost every BI demo I have seen for the last decade. The disclaimer on these demos should also be that their product works great as long as all your data is consistent, comprehensive, clean and current.

• One size fits all. No way. Business people have different degrees of analytics needs and skills, need to use a diverse data types and volumes, and are trying to answer a wide variety of business questions.  It’s short-sighted to assume that a single BI analytical style or tool could solve everyone’s problems.

While it’s true that each new generation of BI tools is more powerful and easier to use, when you get past the hype and look at real implementations, you’ll see that the most pervasive BI remains the spreadsheet (which is often embedded in data shadow systems or spreadmarts).

There’s nothing new here. People have been writing about the limitations of self-service BI for years:

•  Eckerson: Self-service business intelligence not a give-and-go affair, 2013
• The Elusive Promise of Self-Service BI, Sep 14, 2009
• Self-service business intelligence tools failing to make headway at most organizations, Jan 27, 2009
• Moving BI to the Enterprise, Sep 1, 2005

What does the failure of self-service BI mean for Big Data? Big problems. Unless self-service and pervasive BI become the norm, enterprises will never productively tap the business ROI from Big Data no matter how much they invest in it.

If enterprises are to implement self-service BI, whether for Big Data or the traditional data in their enterprise, then they have to understand what it is and what it is not. Once enterprises understand what they are trying to achieve, then they are likely to do things differently to avoid the traps that each wave of self-service efforts have encountered.

Stay tuned, my next few posts will explore these issues in more detail.