datadoodle

Big Data, big hype, big danger

Ted Cuzzillo — Fri, 12 Apr 2013 19:01:20 +0000

A remarkable thing happened in Big Data last week. One of Big Data’s best friends poked fun at one of its icons: the Three V’s.

The well-networked and alert observer Shawn Rogers, vice president of research at Enterprise Management Associates, tweeted his eight V’s: “…Vast, Volumes of Vigorously, Verified, Vexingly Variable Verbose yet Valuable Visualized high Velocity Data.”

He was quick to explain to me that this is no comment on Gartner analyst Doug Laney’s three-V definition. Shawn’s just tired of people getting stuck on V’s.

How strange to be stuck on a definition, but we get stuck all the time trying to define Big Data. Other terms are easier. We’ve always known what visualization is. We seem to agree on “self service BI.” We also know what relational databases are, what ETL is, and all kinds of other established technology. We don’t agree on “business intelligence” or “decision support,” but somehow we don’t dwell on it. We don’t even quibble too heartily with “easy to use,” even though I could argue that we should.

So what is it about Big Data? Is it so much bigger than everything else? I don’t think so. We quibble endlessly and tiresomely because Big Data’s benefits live mostly in the imagination. There are just too many versions of the truth.

It’s as if an emperor went to his royal tailor, got measured up, and — so flattered, he was, and so enamored of the new material his tailor described — he left wearing just the measuring tape and imagined the rest. Outside, his loyal crowds cheered. Soon everyone was certain the emperor really had new robes.

The patient and imaginative among us appreciate the potential. Technology has given us greater ability to manage all our clicks, tweets, and machine effluent. Meanwhile, business users are more interested than ever in what all the new data may tell them. Skeptic that I am, even I see it. I like to make an analogy with television’s emergence and its finer and finer resolution and dimensions.

That idea comes from one of the few presentations I’ve seen at which anyone made real sense of Big Data. Last summer at Scott Humphrey’s Pacific Northwest BI Summit, Harriet Fryman of IBM and Colin White of BI Research described the work in progress with concrete examples. (I wrote about it here on Datadoodle and later for Information Management, here.)

I can wait years for that to develop. It’s the hype and the preoccupation that makes me impatient. Blogs and articles yammer on with the benefits of “big data” when in fact they’re repeating promises made years ago about the benefits of small data and small analytics. This is old decision support super-sized and warmed over, the “new and improved” that won’t satisfy any better than the original but which costs much, much more.

This is where I join visualization guru Stephen Few. Last summer in his essay “Big Data, Big Ruse,” he wrote, “If you’re like me, the mere mention of Big Data now turns your stomach,” and “Big Data is the technological expression of gluttony.” He quoted a book that would be more popular in the industry if concern for analytics and insight were more widespread:

As Richards J. Heuer, Jr. argued in Psychology of Intelligence Analysis, the primary failures of analysis are less due to insufficient data than to flawed thinking. To succeed analytically, we must invest a great deal more of our resources in training people to think effectively, and we must equip them with tools that support that effort.

Similar though less visceral thoughts come from consultant and industry analyst Mark Madsen, one of the most interesting minds in the industry. Toward the end of an early 2011 presentation at Strata Conference titled “The Mythology of Big Data,” he gave the decision support industry a tip:

We succeed only as well as the users of the tools that we provide succeed with our aid. Since most of us are working for or supporting organizations or corporate decision making, that’s the stuff that needs to be supported, and it needs to be supported through proper tools. It’s not just about big and it’s not just about data.

I expect Steve and Mark to watch the emperor away from the crowds. But frustration seems to have gathered in the good seats, too. Gartner analyst Merv Adrian tweeted last weekend, “Enterprises don’t want to buy ‘big data,’ [they] want solutions. If they don’t have [one] or a way to find [one] …, ‘big data’ is a waste of time.”

Even a representative of a vendor that profits well on Big Data technology warns of Big Data fatigue. He says, “There’s a big ‘so what?’ building” among business people. The company continues to push Hadoop, though. He says, “We don’t want to seem like old news.”

One marketing guy who’s beating the “pretty big data” trail of terabytes, not petabytes, sees a “chasm” building from marketing that creates a “special conversation.” It splits technical teams and makes us obsess over “the elite, the power user, the data priest.” SiSense vice president of marketing Bruno Aziza says, “It drives me nuts.”

I admit that I may not even pay enough attention anymore. I may have begun to do what the venture capitalist and influential guy Paul Kedrosky does now: Filter out Big Data. “Soothing,” he tweeted early this year. “Recommended.” Big Data Hype gets shut out, and so does the industry with it. There we have Big Danger.

Collaboration for the collaboratively resistant

Ted Cuzzillo — Tue, 02 Apr 2013 10:01:34 +0000

“Unless you’re a sociopath,” the slogan goes, “you’re going to collaborate.” It works. In one breath, “sociopath” gets our attention while it identifies and strokes us. Then we’re left helpless to disagree with the last part, “you’re going to collaborate.”

Of course we will! Collaboration works for us normal people. QlikTech vice president of product management Donald Farmer’s slogan is so good, in fact, that we accept it without thought. Good thing, too, because collaboration is a wave the decision support industry can’t afford to miss.

The trouble is that collaboration is a tricky one. What makes it especially hard are all the people in business who are technically sub-sociopathic but who are in fact collaboratively disabled. The sociopath wannabes, pretenders, bluffs, intellectual exhibitionists, and other varieties could make the industry long for the good old days when Big Data seemed big.

Even true-blue sociopaths, I suspect, are actually more common in business than we like to think about. We know they’re not uncommon in everyday life, and even they have to make a living. Everyone middle-aged or older has at least one first-hand story of someone in the workplace who may have been one.

In fact, I gather from reading about them on Quora that no matter what their foibles, many these people are highly competent. Imagine this hiring decision: an assimilated sociopath who can deliver on bone-breaking deadlines and sales quotas versus a normal applicant who can’t. Which would you choose? Even the person who set those onerous deadlines and quotas…yes, you guessed it: that manager knows the secret handshake, too.

My closest brush was with a company owner who employed me briefly. He had somehow made his small company survive for years, always at about the same size. He regarded himself as a genius, and every attempt at partnership had failed disastrously.

His mouth even when smiling seemed ready to bite, or rip, lower teeth first. His eyes were wide open, at once enraged and pleading. He was tall, lean, muscular, and slouched forward. Several times a day he came out of his office and walked slowly around the tiny company like a carnivorous fish in a Plexiglas living room tank.

He didn’t collaborate in the way most people do. Under his gaze, people felt like a doomed mouse as it’s pawed ever so gently by a towering cat. “What do you think?” was only a dare to make a move, to try anything at all.

Those who survived there never dared but agree with him, then stay out of sight. One of the first things I noticed were the subdued all-company meetings. In just about every other company, people gathered and chattered before the meeting began. Here, conversations were muted and short.

Let’s say we can avoid such characters. There are plenty of other, “normal” people who seem unable to collaborate.

I recall meetings of one committee or other within a large public-interest organization at which a dozen learned people routinely spent sunny Saturday afternoons inside, chewing over a low-stakes decisions. One hair-splitting point after another was raised and rebutted, then raised again. Frustrated participants went outside to sip coffee among themselves. Would a good tool have told the intellectual exhibitionists to shut up when the moderator wouldn’t?

Then there are all the other varieties false collaborators, the ones who can’t or just won’t engage honestly and wholeheartedly. It’s an everyday event in many organizations.

This is the challenge: The ordinary tools will aid collaboration among average collaborators. The great tool will aid the average collaborator who faces non-collaborators.

How? I frankly don’t know, but I find the possibility fascinating. I suppose that if the would-be collaborator has any leverage at all, the tool might be used to focus the non-collaborator’s attention and then to present the facts beautifully but starkly. It might fascinate, amuse, and challenge. It might lure all participants into asking questions.

Visualized data? Obviously. But is that all there is to it? I’m not done with this one yet.

BI’s “promised land”: bigger than tech

Ted Cuzzillo — Fri, 15 Feb 2013 00:09:12 +0000

At first glance, this pair of tweets last week sounds like a version of BI’s traditional campfire song:

I’ve seen the promised (BI) land, and we are there: databases that fly and process any data; BI tools that are easy to use and fast. Wow! I’d retire but mainstream firms will take 10 years to capitalize on all the new technology & overcome dirty data & politics.

The refrain might go, “When tools fly, they will fly by themselves!” Other lines would caution us to update often, eliminate “politics,” and eat our carrots.

I like to make fun of the industry, but not this particular tweeter. Had he allowed himself a third tweet, he might have mentioned one of his most valuable observations, one for which the industry should be grateful: the “purple people.”

Wayne Eckerson knows all the campfire songs. He’s steeped in the BI industry. He was the director of TDWI Research, and has since moved over to TechTarget. He watches databases and other tools, but unlike many he also appreciates what he calls “the soft stuff.”

He explains “purple people” in the first chapter of his 2012 book, Secrets of Analytical Leaders: Insights from Information Insiders.

Breakthrough innovation occurs when reconciling opposites. Living at one end or the other of a philosophical or cultural spectrum is comfortable, but not terribly interesting or instructive. You know the answers before people ask the questions. Your past, present, and future are hard wired and unchanging.

That sounds like much of today’s conversations in business intelligence.

But people who live at the confluence of disparate approaches and opinions have a broader perspective. They see connections and possibilities that others miss. They speak multiple languages and gracefully move between different groups and norms. They continuously translate, synthesize, and unify. As a result, they imagine new ways to solve old problems, and they reinvent old ways to tackle new challenges. They are powerful change agents and value creators.

In the world of analytics, I call these men and women “purple people.” They are not “blue” in the business or “red” in technology, but a blend of the two, hence purple. Purple people are true analytical leaders …

They are “purple” not just from the blend of red and blue but from the actual blood in their veins, and probably also from the bruises they sustain. They are the new heroes, the new power centers, and the up-and-coming executives.

Tools may fly, and sparks may fly, but these people know better than most what to do with it all.

Buy his book. I hope it helps shift the business intelligence industry’s conversation further away from the purely technical and toward skills, culture, and organization. Secrets of Analytical Leaders: Insights from Information Insiders (2012; Technics)

The wisdom of one in a circled R

Ted Cuzzillo — Thu, 24 Jan 2013 12:30:57 +0000

What does Howard Dresner’s recent trademark registration mean?

He says that registering “Wisdom of Crowds” is “all about protecting intellectual property.” Well, obviously.

It’s obviously more than that, too. This is the guy who’s famed for naming and helping define the business intelligence industry. When he shows up at TDWI and other hardcore BI events, the bright lights of notoriety still gleam on his shiny head. This is Mr. BI, and he’s just trademarked a slogan I’d guess most BI types still don’t get.

He completed the registration in September, according to the U.S. Patent and Trademark Office trademark database. In 2011, he also registered “Wisdom of Crowds Business Intelligence Market Study.”

Over the years, he’s had the good sense to migrate into the human side of BI and performance management. On those subjects, he’s written two books and publishes annual research reports. He also hosts the popular Friday-morning tweetchat “BI Wisdom.”

A trademark registration by itself is small, just a stake in the ground. It’s a just a marker on a new path, and I hope the industry follows his lead again.

Self service BI, dead or alive?

Ted Cuzzillo — Mon, 10 Sep 2012 07:02:30 +0000

A brief stir erupted several Friday mornings ago when Yellowfin CEO Glen Rabie declared to the Boulder BI Brain Trust that self service business intelligence is “dead.” It was one of those statements that makes you sit up and listen — one that leads you all the way to a surprising observation that now drives Yellowfin strategy.

If there had been a death, a body would surely have shown up, at least reports of missing solutions. We would have had to contact the family, starting with Tableau, Spotfire, and who knows who else. It’s a big, busy clan.

But this industry will never produce anything as cold and hard as a dead body. There was no body, and there was no death. And self service BI is no more dead than self-service email, phoning, dressing, driving, or cooking.

I followed up with Glen, whom I knew from the recent Pacific Northwest BI Summit, to find out what’s going on.

He knows self service BI still lives. What he wants to kill is the widespread assumption in the BI industry that self service is what everybody wants.

My rendition of the industry’s dream goes like this: If we could only show those holdout knuckleheads the beauty, the insight, the oh-my-god-I-didn’t-know-that amazement of data analysis, then simply everyone with a brain would sign on.

“Most people who use analytics want to be consumers of a service that is provided to them,” he write to me in email from his office in Melbourne, Australia. “Building it yourself is not convenient.” Google Analytics, for example, is not hugely insightful, but it costs little.

“If we start with the assumption that most people never want to write a report, and never want to do self service analysis,” he writes, “it will force us to innovate and think about alternative ways to deliver BI.”

What about the roaring success of self-service tools?

They’re for data analysts, he says, who’re willing to handle the tricky backend part of data preparation.

“To do BI really well is a special skill,” he said in a call a few days ago. “There are whole books written on what charts to use. You’re just a generalist prior to knowing that.”

Yellowfin discovered this, he said, after trying to serve the self-service market. “We really did believe Yellowfin was for everyone. We thought everyone wanted to write reports,” he said. But they found too little demand for that.

“In reality, customers didn’t want to do data.” Yellowfin began an 18-month shift to “productionized mass distribution of pre-defined analysis.” They’ve broadened the collaboration features, made a more engaging user interface, and built in device independence. In November, they say they’ll introduce storyboards.

We’ll see, but he might be right about the average user’s preference, at least among today’s workforce.

Next questions: How will the specialists be organized within the enterprise? What skills will they have? How will organizations prevent the slothful response that still drives much of the demand for self-service? If Glen’s right, these questions — already the most interesting in BI today — will be even more important.

Data surfing with big data

Ted Cuzzillo — Thu, 16 Aug 2012 21:01:33 +0000

The usual big-data story leaves out crucial bits. We hear about the “what” — big, huge data of all kinds. We hear about the “when” — now and coming soon. We hear about the “how” — Hadoop with helpers. But we almost never hear about the “who” and the “why.” Who’s bothering to analyze all this data, and why?

If we believe big data’s usual, small-bore spokespeople, the whole thing is little more than getting a big enough machine to crunch mountains of data. But if that’s all it is, then all we have is warmed over business analytics. As I endure minutes upon minutes of Hadoop-speak, I’ve often grumbled to myself that if there’s anything more to this story, I sure wish someone would cough it up.

Finally, someone has. In July, president of BI Research Colin White and director of business analytics at IBM Harriet Fryman gave a refreshing presentation at the annual Pacific Northwest BI Summit, held in Grants Pass, Oregon. Yes, big data’s a big deal.

Though Colin and Harriet listed nine conclusions, I derived my own: Big-data analytics can become more than a cost, it can become a profit center and an asset. Second, high resolution is a better way to think of big data’s function than any others I’ve heard.

Unlike most talks, they supported their thoughts with actual cases — the most interesting of which was Sears, the stumbling brick and mortar chain. In April its big data operation opened its doors as a subsidiary to non-competing retailers. MetaScale’s sole purpose is to help find meaning in big data. A cost center has become a new business.

Like most big deals in business, this one echoes the past: the story of Southern Pacific railroad’s phone system. During its long domination of the West and Southwest, it had built a vast telephone system. By the mid ’70s, a few train crews actually had early mobile phones, bigger than a pork loin. Then cash tightened in the late ’70s, and that cost center became an asset when it became Sprint’s foundation.

The data itself should be thought of differently. Big data’s per-unit value is lower and has an inverse proportion of the volume and value. While we groom and pore over transactional data, with big data we throw the stuff around with shovels. Its value is in bulk because it shows value with patterns. Much of that big data, in fact, may end up discarded.

Dare I compare big data to TV viewing? Faced with either one, we may glance, evaluate, and in a blink decide to discard one sample and dwell on the next. With a remote in hand, we say we’re “channel surfing.” A comparable willingness to load data and discard it could change the whole game of analytics. Jill Dyché said, “‘Here’s the data. Go play.’ ‘Because I can’ isn’t a good reason in data warehousing,” she said, “but in big data it’s perfectly OK.” That, she said, is a “game changer.”

To sense big data’s potential, we may again think of television. The early, vacuum-tube powered TV was monochrome and a little weird. A novelty, but a poor substitute for radio. But by the ’60s, the picture had cleared up and by the mid-’60s shows were seen “in living color.” People hurried home to catch new episodes. Today, we’ve got HD on iPads, and the effects are still unfolding. It’s been video all along, but each improvement changed applications profoundly.

If all today’s experts can do is describe big data in terms of tools they know, of course it will sound like little more than new and improved BI. Big data dares us to think much, much bigger than that. It may challenge our tools for the moment, but in the long run it’s a bigger challenge to our imagination.

Data analysts will bridge business and IT

Ted Cuzzillo — Tue, 14 Aug 2012 18:34:44 +0000

Future data analysts will be a “bond” between business and IT, says Diego Klabjan, director of the new Master of Science in Analytics program at Northwestern University. The first cohort starts next month.

Why IT says “no!”

Ted Cuzzillo — Fri, 10 Aug 2012 19:06:34 +0000

Many data repositories are fragile, held together with shaky, ad libbed fixes, observes a busy consultant. Free access for data analysts scares the hell out of administrators.

MacGuffin, anyone?

Ted Cuzzillo — Mon, 06 Aug 2012 18:28:16 +0000

In his years in BI, says QlikTech VP of product management Donald Farmer, “I’ve never seen a project fail for purely technical reasons. The organizational aspect was always the principal cause of failure.” Yet tools remain the industry’s obsession.

Introducing “Sparks”

Ted Cuzzillo — Sun, 29 Jul 2012 04:36:23 +0000

Sparks are barely processed bits that may or may not ignite into a full Datadoodle post or on BI This Week or Information Management.