Adam Gartenberg’s Blog

Interesting results from DB2 BLU vs SAP HANA data mart sizing

Adam Gartenberg — Tue, 28 May 2013 11:31:02 -0400

Paul Zikopoulos recently shared on Twitter some results from a sizing done on a 10 TeraByte data mart, comparing IBM DB2 BLU on Power Systems with SAP HANA.

To summarize some of the primary differences the team found in this particular instance:

HANA requires 4x more RAM
HANA requires 10x more required disk
HANA requires 10x more CPU cores
With HANA , the client would have to buy new hardware and not reuse the hardware they have
HANA would require Intel Westmere processors, not Intel Sandy Bridge processors
HANA must backup to disks and can't back up to Tivoli Storage Manager, Netbackup or Legato
HANA isn't able to offer controls to restrict DBAs without a business need from reading data

You can see more over on the @BigData_paulz Twitter feed.

Trident Marketing Increases Revenue 1,000% with Predictive Analytics

Adam Gartenberg — Thu, 9 May 2013 08:00:00 -0400

A few weeks back I attended the IBM Big Data, Integration and Governance Forum in Charlotte, NC. (If you haven't made it to one yet yourself, we've got more of these events coming up in a number of cities in the US and around the world.)

It's always good to hear our executives speak about how IBM is approaching the area of Big Data, the type of use cases that we're seeing businesses focus on, and learn more about the latest innovations in the fields of Information Integration and DB2 with BLU acceleration. But the highlight for me was when Brandon Brown spoke on behalf of Trident Marketing and the results they've seen from running analytics on Netezza (now PureData for Analytics) along with support from IBM Business Partner Fuzzy Logix.

You can find much of what Brandon shared in the Trident Marketing case study, which describes some of the areas through which they have increased revenue nearly 1,000 percent through their use of predictive analytics.

Operational efficiency is crucial for Trident Marketing, which is a direct response marketing firm that resells solutions from brands such as DirectTV, ADT and Travel Resorts of America. Spending too much on marketing (or on the wrong marketing) and having too high a customer churn rate was holding the company back from hitting their growth targets. After deploying analytics, they saw revenue rapidly increase from the $5M plateau they had been on to $53M in just four years.

Trident deployed analytics in a few strategic areas. One set of analytic models combines data from the company’s telephony, customer relationship management and order entry systems
with data from external sources, such as credit bureaus and clickstream data from Google and Bing. This new level of insight led to a 10% increase in revenue in just 60 days, while also allowing them to reduce marketing costs by 30% be eliminating unproductive search terms or terms that were not cost effective.

But that's not all that Netezza and Fuzzy Logix helped them do:

What is most exciting, says Trident Marketing CIO Brandon Brown, is the use of predictive analytics to help its telemarketers know exactly whom to call, at what moment, and in what geography about what product. It will also enable the company to predict which sales person would have the best chance of closing a particular sale and which products each caller would most likely be interested in.

Check out the full case study for more details.

Link:
Trident Marketing Increases Revenue 1,000% with Predictive Analytics

IBM X-Force Trends: Data Breach Issues and Solutions (Upcoming Webcast)

Adam Gartenberg — Wed, 1 May 2013 08:30:00 -0400

If you are an IT security, compliance and data management professional, you're likely working harder than ever to secure your enterprise against both internal and external threats.

If you'd like to get a better look into recent public vulnerability disclosures and understand the lessons that can be learned from those situations, you'll want to check out the upcoming Dark Reading "IBM X-Force 2012 Annual Trend Report: Data Breach Issues and Solutions" webcast. (May 9, 2pm ET)

Join Robert Freeman of IBM X-Force® Advanced Research as he presents an in-depth analysis of recent public vulnerability disclosures and discusses important lessons learned.

By attending this free, highly informative webcast, you’ll discover:

Five common mistakes IT organizations make when it comes to data security
Why the cloud makes it more important than ever to understand how users access data
How to implement evolving best security practices despite limited IT resources

Webcast attendees will also receive a complimentary copy of the IBM X-Force 2012 Annual Trend Report.

Click here to Register or find out more

Date: Thursday, May 9, 2013
Time: 2:00 PM ET / 11:00 AM PT

Reminder: Big Data at the Speed of Business Broadcast (4/30)

Adam Gartenberg — Wed, 24 Apr 2013 14:37:29 -0400

As a reminder, we'll be holding a broadcast next Tuesday, April 30th, where we'll be discussing details on the recent DB2 with BLU Acceleration and Big Data Platform announcements.

The event will feature General Manager Bob Picciano, VP of Development Martin Wildberger, VP of Product Strategy Inhi Suh, as well as first-hand experiences shared by a client panel and a Q&A session. The broadcast will be held at two times, 11am-12pm EDT and 9pm - 10pm EDT, and you can register here.

Agenda:

Fueling Change in the New Era of Computing - Bob Picciano, General Manager, Information Management, IBM Software
IBM DB2 with BLU Acceleration: Enabling Speed of the Thought Analytics - Martin Wildberger, Vice President, Information Management Development, IBM Software
Client & Partner Experiences with IBM DB2 with BLU Acceleration
IBM Big Data Platform: Advancing Consumability and Performance of Big Data Solutions - Inhi Suh, Vice President, Information Management Product Strategy, IBM Software
Client Experience with IBM Big Data Platform
Question & Answers

Register Today

Announcement Review: IBM’s New Moves in Big Data

Adam Gartenberg — Thu, 11 Apr 2013 10:56:00 -0400

Joe Clabby of Clabby Analytics has posted his first take on the DB2 and Big Data announcements we made last week. You can read his full write-up here: IBM’s New Moves in Big Data: Acceleration, Optimization, and an Open Source Alternative (pdf)

IBM’s New Moves in Big Data: Acceleration, Optimization, and an Open Source Alternative

IBM Almaden Labs, San Jose, CA, April 2, 2013
In a major, multi-faceted announcement, IBM announced yesterday that it will soon deliver “speed-of-thought” analytics using DB2 “BLU” acceleration, improvements in its BigInsights and Stream products, and a tuned-for-Hadoop PureData System. IBM also announced time series improvements in Informix database line. What do all of these announcements mean?

From a big picture perspective, IBM is doing the following:

Dramatically accelerating the performance of its analytics products;
Simplifying the deployment and operations of analytics systems (turnkey analytics systems combined with usability improvements in software);
Creating an “enterprise-class” alternative to open source Hadoop offerings;
Differentiating itself in the smart meter and sensor data analysis markets; and,
Increasing competitive pressure on Oracle and Teradata in the database and analytics markets.

> More

Reminder: Register for the April 30th IBM broadcast to learn more about our recent announcements.

New Ad: Big Data and Analytics

Adam Gartenberg — Mon, 8 Apr 2013 10:22:40 -0400

Here's a new ad on Big Data and Analytics that will be debuting on TV at the Masters golf tournament this week:

More:
Big data and analytics can help predict trends down to the customer. Sales down to the shelf. What can big data and analytics do for you? Learn more at http://ibm.co/10x97Xc

Two opportunities to learn more about the Data Mgmt and Big Data announcements

Adam Gartenberg — Wed, 3 Apr 2013 15:37:07 -0400

If you'd like to learn more about all of the Data Management and Big Data announcements made yesterday, we've got two great opportunities coming up.

First up is a Google Hangout Video Chat this Friday, April 5, starting at 2pm US-EDT. During the chat you'll hear from panelists Jeff Kelly (Wikibon), James Kobielus (IBM), Mike Martin (BTRG) and others. Head on over here to learn more and RSVP.

The second opportunity will be to join us on April 30th for a broadcast event featuring General Manager Bob Picciano, VP of Development Martin Wildberger, VP of Product Strategy Inhi Suh, as well as first-hand experiences shared by a client panel and a Q&A session. The broadcast will be held at two times, 11am-12pm EDT and 9pm - 10pm EDT, and you can register here.

Agenda:

Fueling Change in the New Era of Computing - Bob Picciano, General Manager, Information Management, IBM Software
IBM DB2 with BLU Acceleration: Enabling Speed of the Thought Analytics - Martin Wildberger, Vice President, Information Management Development, IBM Software
Client & Partner Experiences with IBM DB2 with BLU Acceleration
IBM Big Data Platform: Advancing Consumability and Performance of Big Data Solutions - Inhi Suh, Vice President, Information Management Product Strategy, IBM Software
Client Experience with IBM Big Data Platform
Question & Answers

IBM Announces New Innovations to Help Organizations Benefit From Big Data

Adam Gartenberg — Wed, 3 Apr 2013 15:10:18 -0400

As I posted earlier in the day, IBM is making a number of announcements today, including new releases of DB2; PureData Systems; InfoSphere BigInsights, Streams and Entity Analytics; and Informix TimeSeries. What they have in common is that they are all centered around helping organizations benefit from Big Data.

Today's announcements include:

BLU Acceleration, a breakthrough set of technologies, including Dynamic In-Memory technology, that deliver speed of thought analytics over terabytes
DB2 10.5 with BLU Acceleration, flexible, multi-workload database software including 8 to 25x faster reporting and analytics, up to 1000x performance improvement, 10-25x overall database savings, always available transactions and unprecedented reliability
PureData System for Hadoop simplifies the complexities of deploying big data in the enterprise and augments the data warehouse with a queryable archive
InfoSphere BigInsights, Streams, and Entity Analytics provide innovations in our big data platform to improve performance, consumability and exploration of big data in motion and at rest
Informix TimeSeries delivers operational reporting and analytics acceleration for smart meter and sensor data

See below for links to more in-depth technical background, and head on over to DeveloperWorks for a free trial of DB2 10.5 with BLU Acceleration.

Today's press release provides more background on the various elements of these announcements:

BLU Acceleration enables users to have much faster access to key information, leading to better decision-making. The software extends the capabilities of traditional in-memory systems -- which allows data to be loaded into Random Access Memory instead of hard disks for faster performance -- by providing in-memory performance even when data sets exceed the size of the memory. During testing, some queries in a typical analytics workload were more than 1000 times faster when using the combined innovations of BLU Acceleration.
Innovations in BLU Acceleration include "data skipping," which allows the ability to skip over data that doesn't need to be analyzed, such as duplicate information; the ability to analyze data in parallel across different processors; and greater ability to analyze data transparently to the application, without the need to develop a separate layer of data modeling. Another industry-first advance in BLU Acceleration is called "actionable compression," where data no longer has to be decompressed to be analyzed.

PureData System for Hadoop is definitely something you should explore if you are looking at including Hadoop in your enterprise deployment:

The new system can reduce from weeks to minutes the ramp-up time organizations need to adopt enterprise-class Hadoop technology with powerful, easy-to-use analytic tools and visualization for both business analysts and data scientists. In addition, it provides enhanced Big Data tools for monitoring, development and integration with many more enterprise systems.

Also included in today's announcement are updates to the following products:

A new version of InfoSphere BigInsights, IBM’s enterprise-ready Hadoop offering, which makes it simpler to develop applications using existing SQL skills, compliance security and high availability features vital for enterprise applications. BigInsights offers three entry points: free download, enterprise software and now an expert integrated system, IBM PureData System for Hadoop.
A new version of InfoSphere Streams, unique “stream computing” software that enables massive amounts of data in motion to be analyzed in real-time, with performance improvements, and simplified application development and deployment.
A new version of Informix including TimeSeries Acceleration for operational reporting and analytics on smart meter and sensor data.

In Depth View
If you'd like a more in-depth look at the technology being announced, head on over to the online IBM Data Magazine, where you'll find articles on DB2 10.5 with BLUE Acceleration, Informix 12.1, and the new PureData System for Hadoop.

Additional Coverage

More from IBM on Big Data and Data Management

Information On Demand 2013: Call for Speakers

Adam Gartenberg — Wed, 3 Apr 2013 10:36:52 -0400

Session proposals for this year's Information On Demand conference will be accepted from April 4th - 30th, so if you have an interesting story to share about products, solutions, or implementations, please head on over and submit a proposal.

IT professionals, including managers, architects, and developers should submit proposals for technical sessions across Information Management, Business Analytics and Enterprise Content Management. C-suite executives and senior decision makers in Line of Business and IT management roles should submit proposals about industry-related thought leadership and innovative ideas for Business Leadership sessions.

You can find more information at ibm.com/software/data/2013-conference/speakers, and for everyone else, remember that registration opens on May 15.

ibm.com/events/informationondemand

NY Times: Why Big Data Is I.B.M.’s Friend

Adam Gartenberg — Wed, 3 Apr 2013 10:18:08 -0400

The Bits blog in today's NY Times highlights some important news on Big Data that IBM is sharing today while also describing why Big Data is so important to the businesses that IBM works with (and why IBM in turn is putting so much investment into related technology.)

Case in point: I.B.M. just announced a series of technologies intended to make data analysis faster and more powerful. That is probably good news to Big Blue’s core customer base of large corporations, which are increasingly drowning in digital information from both the Web and the physical world. A good portion of that data is already derived from and running on I.B.M. technology.

I.B.M.’s offerings include an easier way to load corporate data into an I.B.M. machine and produce analysis quickly; an analysis “accelerator,” which can speed the production of analytics reports, I.B.M. says; and a Big Data “appliance,” or combination of hardware and software, that enables companies to posit questions, then have them answered automatically when the appropriate data comes along.

Bob Picciano, General Manager for IBM's Information Management business, highlights the type of performance improvement that will now be possible, such as running queries that might previously have taken two days in only three minutes. The article cites just one example of this type of improvement: "A customer cited by I.B.M., BNSF Railway, was said to have had a 100-fold improvement and a 90 percent drop in data storage consumption using the new technologies."

You can read the full article at the link below, and check back in here for further updates throughout the day.

Links:

From Big Bang to Big Data: ASTRON and IBM Collaborate to Explore Origins of the Universe

Adam Gartenberg — Thu, 14 Mar 2013 11:55:10 -0400

"The SKA’s [Square Kilometer Array's] more than 500,000 antennas scattered across southern Africa and Australia will produce a torrent of data that’s equivalent to 10 times the daily global Internet traffic."

IBM continued to push the push forward the cause of Big Data this week with an ambitious initiative: better understanding the origins of the universe. It's one that will require the latest approaches to extracting value from big data.

The quote above comes from a post by Senior Vice President for IBM Research, John Kelly (The Square Kilometer Array: The World's Ultimate Big Data Challenge)describing how IBM will be working with ASTRON, the Netherlands Institute for Radio Astronomy, and South Africa on the DOME project, a multi-year partnership funded primarily by the Dutch government aimed at developing an information technology system for harvesting insights from the SKA’s data.

You can read more about this initiative in the white paper, The Square Kilometre Array: The Ultimate Big Data Challenge and the press release from earlier this week: From Big Bang to Big Data: ASTRON and IBM Collaborate to Explore Origins of the Universe, or watch the video below.

How does the work above apply to every day business problems? A recent study of over 1,100 business conducted by the IBM Institute for Business Value and the University of Oxford Said Business School explored just this question. The study, titled "Analytics: The real-world use of big data. How innovative enterprises extract value from uncertain data" uncovered where businesses are in planning their big data deployments and other common trends and insights, including:

Across industries, the business case for big data is strongly focused on addressing customer-centric objectives
A scalable and extensible information management foundation is a prerequisite for big data advancement
Organizations are beginning their pilots and implementations by using existing internal sources of data
Advanced analytic capabilities are required, yet often lacking, for organizations to get the most value from big data
As organizations’ awareness and involvement in big data grows, we see a consistent pattern of big data adoption emerging.

You can read an executive summary of the study here (download) or register for the complete results.

For more on how smarter analytics can transform big data into big results for your business, please head over here.

The data on Big Data is, well, big

Adam Gartenberg — Wed, 5 Dec 2012 14:54:38 -0400

As an infographic over on the IBM Big Data Hub highlights, we're now at the point where there's a lot of data about Big Data. (And if you haven't checked it out yet, the hub is a great place to look when you are trying to cut through all the noise.)

Walmart processed 5,000 transactions/sec at peak time Black Friday

Adam Gartenberg — Tue, 27 Nov 2012 09:14:18 -0400

I'm just going to leave this here. You'll either know why or you won't. :)

Wal-Mart: 22M+ customers served since opening stores last night, with 10M register transactions and 5,000 items/sec. processed at peak time.
— CNBC (@CNBC) November 23, 2012

Debunking Myths about Data Scientists

Adam Gartenberg — Mon, 19 Nov 2012 15:37:08 -0400

Over on the IBM Big Data Hub (and in the entertaining slideshow below), James Kobelius debunks the top 10 myths that many people have around data scientists.

Myths and Mathemagical Superpowers of Data Scientists from TheSocialPitt Pittman

Link: James Kobelius - Data Scientists: Myths and Mathemagical Superpowers

Live Blogging the IOD 2012 Day 2 General Session.

Adam Gartenberg — Tue, 23 Oct 2012 09:40:06 -0700

Live Blogging the IOD 2012 Day 2 General Session.

Steve Mills, SVP and Group Executive, Software and Systems
An Explosion of Data

Facebook processes 500+ Terabytes of data daily
At&T transfers about 30 Petabytes of data through its network daily
For every session, NY Stock Exchange captures 1 Terabyte of trade information
150 Exabytes - global size of "Big Data" in Healthcare
420M wearable (wireless) health monitors by 2014, up from 12M in 2012

The good news, and the reason why so much buzz around big data, is that the cost of computing has come down. The incredible quantities of data that we can all touch is

What you've seen from us is a flexible platform. Modularity, open standards. You don't throw away what you have, you use what's there. The file systems you have, the data systems you have, but bringing it together better, federating better, taking advantage of streaming technologies... and using them as base enablers to bring more and more analytic capabilities on top.

Multiple, logical points of entry when you think about the types of projects that will be important to your business.

New Data, New Solutions, New Business Outcomes
From Traditional (e.g., Negative contact center or self-serve experience; pay and chase fraudulent claims/benefits) to New Value (e.g., next best customer action, preventative fraud detection).

Example: Contact Center experience influences customer satisfaction
Next year will be delivering a version of Watson specifically targeted at call center. Use as basis to build out knowledge base so that a new call center agent will effectively have the same knowledge as an experience call center agent.

Airbus: Reduced support call times by 70%, from 15 minutes, saving $36M annually. Bringing in more engineering, other data into the call center.

Fraud is major issue for government agencies. Big Data Opportunity:

In 2011, 235M individual tax returns were filed in the US
US Internal Revenue using social media to investigate tax fraud
Data produced per hospital up from 167 TB to 665TB by 2015
Satellite imagery used t omap location of undeclared assets

Benefits:
Major government medical and social benefits agency:

Identified an improper payment level for a particular benefit of over 40%, worth over $140M
Performed analysis in hours instead of weeks
Ad-hoc analysis of over 70 data sources

NY State Dept of Taxation and Finance

Decreased revenue drain due to improper payments of $200M annually; $1.2B since inception

Energy & Utilities Data Sources

500 million meter readings per day in a typical smart meter project
80% data growth from instrumented smart grids, weather forecasts, documents
Pacific Northwest National Lab - 15% decrease in overall peak loads; consumers saving 10% on their electiricity bills
Oncor - cost savings of $176M

A lot of this can come from taking the data we have, taking the knowledge that we have, and doing it at a scale that's now possible.

No longer constrained by technology, by affordability... what we're often constrained by is the will to tackle these challenges. At IBM we're trying to lead th eway. Broad based, modular, open standards based portfolio. And we want to work with you - get together with you to understand where the art of the possible lies. How to get a rapid return on investment. That may take a matter of months to get implemented and a matter of months to deliver a return, and then iterate, iterate, iterate.

Fred Balboni, Gloal Leader, Business Analytics and Optimization, IBM GBS
Int he world of data, it's no longer what you know, but what you're going to do with what you know, and how fast you're going to do it.
Telco example - identifying who most valuable customers are and who's most likely to leave: 6B call detail records generated in a day. That's big data. that data was always there - they collected it, summarized it, and archived it. So what's new? 1. Using advanced technology to create insight on that data. 2. Taking action on that insight.

Analytics no longer for select few, and no longer isloated in IT.

Talked to clients, some stories:
CMO: My success is no longer about hanging out with the advertising agency; my success is about analytics and technology.
CFO: I am no longer the scoreboard operator for the company; I've become the company's coach. My job is to make suggestions on the plays we should be running.
IT: Job still about technology, but it's about creating new partnership models with the business.

Our challenge is to transform our jobs and the way that we interact with the business so that we can all deliver on the promise of analytics.

A decidedly optimistic agenda: What's my greatest possibility? That is the opportunity.

Moving from existing problems to creating next, greatest opportunities.

Client Discussion
JP Morgan Chase: SVP and head of customer data analytics- Adam Braff
Thomson Reuters - Global head of Corp Systems - Jerry Hope

Q: How is customer changing, and how is bank changing to respond to that new customer
Braff: Don't have new customers, but have new customer behavior - reaching out electronically, on Web. And at same time, not have new banker, but new regulator (e.g., change regulations on Debit card charges changes profitability picture).
Q: How changing?
Braff: How mine data, understand which customers interact in new channels and why not? Are they enjoying the new experience. Feedback loop on what learn in digital channels and use it to fight fraud, contain costs, improve customer experience.

Q: Thomson Reuters has grown so quickly through mergers and acquisitions; portfolio of autonomous business. How leveraging for growth
Hope: Acquire 40-60 businesses/year. Seeking solution for customers coming in through acquisitions, including ones that might already be using Thomson Reuters solutions. Customer 100 program - look at top 100 customers and drive value for them by packaging up solutions that might be offered by acquisitions, but also how drive value back possibly in a different way.
Q: How applying technology?
Hope: Establishing Customer Master - 360 degree view of customer. Focus on customer centricity. Looking at analytics to drive cross sell and upsell. Also looking at how drive down total cost of ownership. Rationalize applications using, already down from 900 to 740 - eventually 300-400 range.

Q: How is the business adapting and becoming more analytically minded to take advantage of these technologies
Braff: Disparate data sources, data from third parties,and putting it to new uses. Also need new ways of vizualizing and making it available to executives who used to use big reports and now want it on an iPad. Requires new talent and skill sets
Hope: Looking to blend tech and uses of it, too. Historically doing a lot of analysis externally, and now looking to internalize it for internal use. Look at how provide more solutions, more real time for users - make more informed decisions on marketing and finance. Also working with IBM on websites. Brought together CMOs from various businesses to take portfolio of 1000 websites and drive down to more manageable platform.

Q: Takeaways for audience - When you work with IT, what would you say are the three things that the business has got to get right to make relationship most effective with IT?
Braff:
1. Recognition of changing regulatory environment; not just about pricess can charge, but way that FTC looks to see if can use 3rd party web data or aggregators of data. Need flexibility on whether can take data sets or come up with own internal data.
2. Need nimble R&D spaces to take new data sets from outside, put it next to existing data, and see if helps us. Speed to market is essential given volatility in market.
3. Once figure out attributes of what's important to business, need help from IT business to deploy them quickly.
Hope:
1. Establishing what business drivers are, the key value is that we're bringing forward. Establish business champion. Bring business users together with IT to come up with solution together.
2. Establish governance - more concept of centers of delivery [like centers of excellence]; within BI have established deep partnerships with marketing and finance to deliver solutions that deliver value
3. Define what expected outcomes are. Common goals and working with business on what objective is to meet ultimate outcome that they expect.

Les Rechan, GM for Business Analytics Software
Harvard Business Review - data scientist is sexiest job of the 21st century.
With data being the new natural resource - or job is to refine that and turn that into insight.
Technology is the starting point... smarter analytics embedded in the business is the silver thread.
Client example: ufone (Pakistan) - Unica, SPSS, Cognos at point of impact to drive 100% improvement in response with better models, and campaigns targeted to customers' unique tastes

Farhana Alarakhiya
You need to listen to customers, communicate to them in compelling, timely matter.
Create rich customer profile and singular approach to decisioning to create experience that will engage and delight customer will driving bottom line improvement.
Next Best Action solution - create rich insights to build service that is fine-tuned to a customer's specific needs. Bring together offer optimization, lifecycle clustering, channel optimization, response modeling, etc. Bring together into a single decision engine. Ensures that all interactions with customers are coordinated and optimized. Enables to sift through millions of subscribers to know who to contact and with what action.

Example:
Scenario: Telco - Market saturation has resulted in telecoms trading customers back and forth; very competitive.
Customer with a number of dropped calls, and also called a competitor's contact center.
What action should they take? Offer discount on this month's bill? Deliver an apology? 6 months free unlimited data plan? Upgrade phone?
Action: Based on what know of Bill, offer upgrade to new phone... also reach out in the evening and via e-mail. Included in email are recommendations of three phones he's most likely to like, and because know that he likes to do research, provide details right in the email.
Also know that he's indecisive (for the last offer, he got halfway through and then abandoned the session). To make sure that doesn't happen, propose video chat before he can leave. Call center agent has full view in front of her on background on him, and list of prescribed actions to take with him. (e.g., acknowledge the # of dropped calls, explain that due to cell tower under repair). Offer discount for that, and then move on to discuss his new phone. When get to store next day to pick up new phone, because they know he has had to call for help setting up phone in the past, they know to offer to help him set it up.

Les Rechan
Examples of how companies across many industries
McCormick - Optimizing business across 30,000 SKUs.
State of North Carolina - Identified $200M in suspicioius Medicaid claims. Review and prioritize tens of thousands of providers and hundreds of millions of claims
Elie Tahari - 97% accuracy in predicting demand. Doing predictive forecasting on customer demand 4 months in advance.

Tom Doub, PhD. CEO Centerstone Research Institute (mental healthcare institute).

Serve 70k people a year recovering from mental health and addiction issues
Frist step - pull together resources and pull together center of excellence for analytics.
Looked for areas where could get small wins. Community based services - staff going out to schools or homeless shelters or homes. Were only getting paid for about 75% of what did. Created tools working with the business to create data transparency, so managers could find opportunities to improve billing. One manager - early adopter/champion. Got such terrific results in short time, that everyone else got on board. Billings over 6 months improved from 75% to 95% .
Looked at access - how quickly can someone get in to get care. Through data transparency, could find inefficiencies in system to improve access to timely services by 35%.
Responding to change. State of Indiana changed about 20 different regulations at once. Had hard time determining which to focus on. Created what-if modeling tool and applied what-if scenarios to each of 20 changes and enabled to see that if didn't make changes, would face 50% revenue loss. Through modelling, identified that there were 3 areas that really had to be concerned about. Once looked at data could also see opportunities provided by changes to transform care. Statewide, care providers lost 26% of revenue. Centerstone only lost 4% of revenue in this area, while also serving more people.
Personalized medicine. One size doesn't fit all, even for same diagnosis.

Art of possible in terms of innovation, partnership between LOB and IT, outcomes and impact. Jobs are sexy and if do it well, help enterprise acheive results.

In main tent, announced that will give away Honda Insight. Go to https://www.analyticszone.comto download Cognos Insight. Create best application, win the car.

On Strawberry Pop-Tarts and Hurricanes

Adam Gartenberg — Mon, 22 Oct 2012 15:11:27 -0700

What do Strawberry Pop-Tarts and hurricanes have to do with each other? Big Data knows...

(via the IOD Blog)

Infographic: Big Data - The New Natural Resource

Adam Gartenberg — Mon, 22 Oct 2012 14:55:53 -0700

A new big data infographic out of the IOD Conference today:

Highlights:

Big data is growing fast; structured and unstructured data is growing at an annual growth rate of 60%
1 in 3 business leaders make critical decisions without the information they need.
53% of business leaders don't have access to information from across their organizations required to do their jobs
Organizations applying analytics to data for competitive advantage are 2.2x more likely to substantially outperform their industry peers.
Organizations adept at analytics enjoy 1.6x more revenue growth, 2.0x more profit growth, and 2.5x more stock price appreciation than their peers.

Live Blogging the Information on Demand Conference General Session

Adam Gartenberg — Mon, 22 Oct 2012 09:37:02 -0700

Live Blogging the IOD 2012 General Session. (I'm posting my rough notes for now, and will look to update it later.)

Host Jason Silva:

Co-evolution of humans and technology.
Accelerated evolution.
Convergence of Big data, analytics, social , mobile, cloud.
90% of the worrld's data created in past 2 years alone.
We have a voice, and we leave a digital trail.
We can see patterns that we cold never see before. We can spot trends as they're happening.
"We're approaching a new renaissance... and all you have to do to participate is to think big."
"We have the tools - all we have to do is imagine what could be."
Digital nervous system - Make organization's feedback loop entirely digital.
"Big opportunities to transform businesses, industries, and the world we live in. This, my friends, is an upgrade."

Robert LeBlanc, SVP, Middleware Software
Smarter Analytics in the New Era of Computing

Study of CEOs - 1800+
What factors are impacting organizations - Since 2004, technology has risen to #1 factor (now above people skills, market factors, globalization.)

CIO survey - What's driving innovation and market growth? Big Data Analytics (83%), followed by Mobility - 74%, Cloud 68%.
Aren't seeing differences by industry or geographic location. Used to see different factors in US, for example - no more.

We are embarking into new era of computing.
Three V's

Volume: T-Mobile: 17 billion events a day. Want to be able to analyze it in real time to provide better service to their clients
Velocity - Dublin City Council. Uses real time data in transportation vehicles (bus, train), providing 50 updates/second. Allows them to adapt in real time, now one of most efficient transportation systems in the world
Variety - Vestas (windmills). Analyze 2.8 petabytes of data to make sure are placing windmills in the right place.

We've seen this before.
1960's - all about transaction systems (banks going online, airline reservations)
1990 - eBusiness - changed the way we thought about applications and provide transactions
2010 - Analytics - Not just producing info, but how gather it all up, provide platform to take all sources of info and process it and get insight. The outcome you want is insight from info. If you can't get info and can't get itt in real time, insight won't be as good.

IBM Big Data Platform
Often think of Big Data as Hadoop. Hadoop is a part of it, but it's much more than that. Content in ECM systems, data warehouses. And you want stream computing because you want to act and react in real time, not after the fact.

Dr. Phil Anno, Prinicipal Scientist, ConocoPhillips
Operations in 30 countries, largest producer of oil and gas north of the arctic circle.
US Based multinational energy company headquartered in Houston, Texas. 16,000 employees worldwide.

About the Big Data project involved in:
About 25% of earth's remaining conventional oil and gas reserves reside in the arctic, but presents many challenges due to remoteness of arctic region and short summer region.
ConocoPhillips has invested over $500B in region. Just one exploratory well will cost around $350M in the arctic region.

Industry will require computational system for real time tracking of ice flows over the region. Safeguard personnel and resources.

With IBM have developed satellite driven real time computational technology . Near term - extend drilling season of summer by 1 month. Drill 2 wells per summer rather than just one, and save mobilization cost on second well of around $300M per drilling season. Longer term, with real time predictive capability, can influence design criteria for production platforms to come. Can plan ice management operations to deploy tugboats, icebreakers to manage the ice around facilities. Eases requirement to design production platform to withstand maximum impact. Savings estimated to be as high as $1B in savings per production platform.

Hurdles that had to overcome:
Challenge was to track all arctic ice flows across all high arctic leases. Like hurricane tracking, but can be several hundred ice flows moving over the path at a given time, each with own path and own cone of uncertainty. This tracking technology, will generate around 1 terabyte of data a day. Developed system of differential equations governing motion of ice. Can generate accurate forecasts of where ice will go.

What pieces had to use to build the solution:
Chose to build the solution on the InfoSphere Streams platform. Very attractive as off the shelf, flexible stream computing platform and development platform. Developed a lot of proprietary application software, but not in the app software business. Streams allowed their subject matter experts to focus on the tracking and analytics modules and take for granted the stream infrastructure for real time data flow. Could also use agile software development methodology. Due to team effort of partnering with IBM, completed it in just 5 months. 4 months of R&D and 1 month of dedicated testing.

What are you looking at next?
Next step - look to operationalize this technology. Add some key additional data streams to constrain the forecasting solution. New data acquisition systems are being engineered as could speak. Looking for additional applications outside of the arctic. For example, real time analytics on drilling, trading of electric power and natural gas.

Robert LeBlanc
IDC: You will spend 68% of your IT budget just on operating costs. (Up substationally from 1996). How free up time and money to focus on challenges in front of us.
IBM Study: Only 1 in 5 organizations allocate more than 50% of IT budget to new projects.

Change how we think of IT so we can free ourselves to focus on big challenges in organization. That's why we came up wtih PureSystems. Our clients are spending too much time building out infrastructure, especially when 80% of time is spent on common tasks.

Build out systems and bring our expertise and your expertise together.

Integrated all the hardware, from the compute to the storage to the networking, all in the box and pre-integrated. On the software side, we said why don't we prieintegrate all of the middleware. All cloud enabled (can virtualize and provision).

And we know you use multiple tools to manage it all. So built single integrated management console.

Free up your time so you can be innovators, rather than building out that next system. Want to invert the %s so you can spend 68% of your time innovating.

First two:
PureFlex - Delviering Infrastructure Services right out of the box
PureApplication - Dleivering Platform Services

Next: PureData System - Optimized exclusively for data services.

Why did we do this? What problem did we think we could help you solve?

Inhi Cho Suh, VP, Info Mgmt Prod Mgmt and Strategy
What challenges: Big Data, Cloud, Mobile are placing a lot of pressure on data systems not equiped to handle them. Companies spending time trying to deploy and tune them, rather than delvering solutions.
How fit into the rest of the family: Has common themes: integration by design, built in expertise, simplified experience.
For example, can build patterns, and deploy database.

Addressing the data challenge with IBM PureSystems
What companies want to do is deliver values back to business with greater speed and consistency.
When talk about data load ready in matter of hours. From cabling to integrated compute capabilities, to firmware upgrades - all done in one process.
Data workloads - transactions and analytics.

An e-commerce system is fundamentally different from a customer analysis system, and that in turn is different than a fraud detection system.

Announcing 3 models of the PureData system - different computer sicence techniques to optimize each ssytem
PureData for Transactions - optimized exclusively for data transactions - integrated DB2 PureScale, compute memory and storage for entire lifecycle of system. Can consolidate over 100 database workloads into 1 system. Savings in compression, storage and speed are amazing
PureData for Analytics - optimized for analytic workloads. Powered on Netezza technology. 20x concurrency and throughput for tactical queries. Including industries richest set of in database analytics
PureData for operational analytics. Optimized for operational analytic data workloads. Integrated InfoSphere Warehouse 10. Can run complex data analysis even while having thousands of concurrent operational queries running at same time.

Robert LeBlanc
One of questions asked in CEO study is new complexities are facing? Responses: We need better information and insight; the time available to capture, interpret and act on info is getting shorter and shorter.
Does analytics create a competitive advantage?
63% percentage of respondents who cited a competitive advantage
3.6x: Organizations competing on analytics are more likely to outperform their peers.

Company that embodies smarter analytics in era of big data computing. 2600 members over 86000 non-acute health care sites. Support over 400,000 physicians. Award winning technology that llows members to

Keith Figlioli, SVP, Premier
Represent 40% of the overall healthcare market. Focus on performance improvement activities to drive high quality and cost effective care using technology.

What challenges face?
US Health care market going through a major transformation. Putting in place new workflow systems, but are siloed and information poor. Very few systems connect data across provider settings. Or even think about population health (e.g., all diabetics in community.) Ex., how many of us have gone to doctor's office and filled out same paperwork, only to go to imaging center and have to fill out all over again.

Providers need advanced analytics today. focus on the heart of that problem.

2 years ago teamed with IBM to build next generation platform, and launched it last summer.

At core of what trying to do:
1. Connect our data
2. Connect our knowledge base
3. Connect our people

Build out next gen data warehouse that includes data warehouse, social business, and workflow to drive right data and process to right people at right time.
Process over 2.5 million clinical transactions in real time. 1 out of 4 patient discharges.

View across quality, safety, labor and overall population health.

Can also run large scale industry collaboratives that can lead to industry best practices. Have become national entity to drive large scale business networking, which is directly integrated into platform, rather than separate entity.

What value do members derive and what are the big payoffs?
Value drivers:
1. Leveraging integrated payer and provider model. Have integrated waste report that shows improvement opportunities. Est. that $750B (almost 30 cents per dollar) wasted on unneeded care, complexity, fraud, etc.
2. Collaborative - Members have amassed thousands of protocols to drive safe and efficient care - real world patterns rather than trying to do in trails.
3. Real time data collection and predictive rules and alerting. Predicts variance before it occurs. 1 out of every 20 patients in US healthcare will get infection while in hospital. 100,000 die each year. One of nation's top 10 causes of death.

Challenge of implementing core transaction systems while also building surveillance system at the same time. Other industries can focus on surveillance because basic transaction system is now there (built out over past decades). Healthcare still catching up there. Providers have no choice but to do both simultaneously.

Robert LeBlanc
How are companies already leveraging analytics to optimize outcomes?

TridentMarketing - 10x increase in revenue by growing retaining and satisfying customers
ConAgra Mills - 5% increase n production capacity
conEdison - $3M savings by decreasing financial risk
Omnicom Group - $200M increase in cash flow by transforming financial processes

Deepak Advani, VP of Business Analytics
Innovations in Smarter Analytics

All data sources, all information, all perspectives (not just pass, but what's happening right now and what's likely to happen). Need to poptimize for all decisions and for all people in the organization.

Integration of data that's on premises and that's outside of the organization. Analyzing data at rest as well as data in motion. Integrating data and analytics... analytics moving to where the data is and not the other way around. A lot of the data is unstructured, it's text, it's multiple languages. Need to understand cynicism and emoticons. Need to understand culture differences. No company that has invested as much in natural language processing as IBM has.

Working with customs and border patrol to take pictures of all license plates crossing the border to analyze patterns (how often are they crossing), and bring that together identity analytics (are they who they say they are, are they a convicted drug trafficker). If you're going to decide to pull that car over, need to decide it in real time.

Decision Optimization: Business Rules + Optimization + Predictive Analytics. Launched earlier this year. Need to have intelligence embedded at the point of decision in the business process. So a call center can make offer without knowing a thing about data analytics.

Analytics for Everyone: Novel visualization. Can design what want visualization in hours, not days. Cognos Insight: Free. Can go to analyticszone.com and download a version of Cognos Insight for free. Boeing - Over 30,000 users on Cognos. They have the dashboard on 52" touch screens so that the people on the factory floor can access it.

Analytics in the cloud: What we are seeing is that having private cloud-based analytics service is working out well. Organizations that have centralized center of excellence for analytics tend to be far more advanced in their analytics journey. We're packaging up private cloud to make that easy and to share best practices from our experience and those of our clients. There is also a lot of need for a public cloud for applications and solutions.

Demo
Running marketing for an airline. Dashboard: Flight delays, revenue over time, passengers over time both declining. From overall sentiment down to integrated view down to customer level. Know preferences so can make more relevant offers (e.g., they tend to travel over school breaks, etc.)

Want to keep track of not only transactional data, but behavioral data, attitudinal information, etc.

Robert LeBlanc
Three things to remember as this week:

Leverage analytics as a game-changer
Embrace big data to maximize your business outcomes
Get inspired and take action.

Think Big: Your Business Depends on it

Dave Laverty, VP Marketing Information Management

A few of the announcements spanning portfolio:

Accelerate value from big data: InfoSphere Data Explorer
Protect your information lifecycle and reduce risk: InfoSphere Guardium v9
Improve IT economics: PureData System
Enable all people to make better decisions: Analytics as a Service, Free Personal Edition of Cognos Insight
Deliver actionable insight: ManyEyes (Advanced Visualization); Risk Analytics (Basel III and Solvency II)
Leveraging market-leading services and research: 2012 Big Data Study
Improve outcomes with content solutions: Patient Care and Insights; Intelligent Investigation Manager; Defensible Disposal

IBM Addresses Security Challenges of Big Data, Mobile and Cloud Computing

Adam Gartenberg — Fri, 19 Oct 2012 09:40:53 -0400

Yesterday IBM announced ten new products and enhancements to help companies deliver real time security for big data, mobile and cloud computing. The announcement, the biggest since the launch of the IBM Security Solutions division earlier in the year, includes the release of IBM InfoSphere Guardium 9.0 software. which now features support for Hadoop and other big data environments..

From the press release:

IBM’s new software capabilities help clients better maintain security control over mobile devices, mitigate internal and external threats, reduce security risks in cloud environments, extend database security to gain real-time insights into big data environments such as Hadoop, and automate compliance and data security management. Along with IBM Security Services and IBM’s world-class research capabilities, this set of scalable capabilities supports a holistic, proactive approach to security threats spanning people, data, applications and infrastructure.

While I'm sure we'll be hearing more about Guardium and related security solutions at IOD next week, here are the highlights of what's in the new release:

With today’s announcement, IBM is among the first to offer data security solutions for Hadoop and other big data environments. Specifically, Guardium now provides real time monitoring and automated compliance reporting for Hadoop based systems such as InfoSphere BigInsights and Cloudera. With federated controls across data sources, clients can understand data and application access patterns help prevent data leakage and enforce data change controls. Built-in audit reporting can be used to generate compliance reports on a scheduled basis, distribute them to oversight teams for electronic sign-offs and escalation, and document the results of remediation activities. In addition, IBM offers data masking to de-identify sensitive data as it moves into and out of big data systems.

Full Press Release: IBM Addresses Security Challenges of Big Data, Mobile and Cloud Computing

In their own words: the benefits of PureData Systems

Adam Gartenberg — Tue, 9 Oct 2012 16:46:30 -0400

If you'd like to hear first-hand details on the benefit organizations are seeing from the PureData Systems, check out the videos below.

The first one highlights the impact the systems are having on Suntrust Bank, BNSF Railway, Premier Healthcare Alliance and NYSE Euronext.

"It's allowing us to get to market faster.... What we're getting from IBM is that they're building the technology, integrating that together for us, and allowing us to worry about building the solutions that our alliance needs. - "Todd Wilkes, Vice President, Enterprise Solution Development, Premier Healthcare Alliance

The second video features NYSE Euronext, a global trading and technology provider, that is analyzing two terabytes of data a day across trading markets.