a) Typical “Big Data” examples within organizations

b) Real-world value propositions from harvesting “Big Data”

Simple Definition and Context

Let’s start with establishing a definition and context for “Big Data” since the name alone can be misleading.  Big Data is a reference to the very large scale of data available or being created that cannot be easily handled using traditional processing, methodologies or technologies.  Big Data can relate to structured or unstructured data.  The challenges with Big Data can include how to capture it, the methods for storing it, how to understand it, how to analyze it, how to search it, or how to visualize it and correlate it to something familiar.  Data can be tagged as “Big Data” by evaluating it against the above classifications and by considering at least two factors:

1) The scale and sheer volume of the data in comparison to what is reasonably expected

2) The speed at which it is created or is expected to grow

Since there is a lot of subjectivity in defining “Big Data”, let’s list some real-world examples to solidify the understanding.

Big Data within Organizations

Using the definition above as guidance, a growing number of possible examples may belong in the “Big Data” category. For the sake of being concise, the following represent the types of “Big Data” organizations commonly encounter and the value that can be derived from them.

Call & Contact Details

Transaction Details

Web Clicks & Logs

Other interesting Big Data examples you may encounter include:

Application Logs

Informational, warning, error, monitoring and event messages are continuously produced by software systems, hardware devices and application platforms.  Proactively recognizing potential issues from the patterns can help improve the quality of service and reliability that the IT groups need to ensure.  Furthermore, it can be an element of a good risk mitigation strategy if the services and platforms are a critical part of your business.  This content is often overlooked for the value it possesses.

Social Media

Tweets, Facebook and Google+ posts, blogs & responses have quickly become acceptable means of social interaction between people.  The sheer number of people using these channels creates a “Big Data” problem.  The data and growth is exceptionally large to deal with. The content is text-based and needs to be evaluated in context to derive at the right interpretation. Furthermore the relevance of the content (eliminating noise) is difficult to decipher.  The jury is still in deliberation over the ROI for harvesting this information.  Nonetheless, it’s difficult to ignore.

GPS Trace Records

Equipment, products and personnel are increasingly fitted with GPS technologies that can track every move from point A to point B.  The ability to proactively analyze this movement can lead to supply chain efficiencies, human capital effectiveness, bottom-line cost reduction, fraud mitigation and allow for overall control and continuous visibility.

Technology & Instrument Output

Needless to say, there are countless unique examples within industries.  Utility and communication companies produce incredible amounts of usage details that can be used to manage demand and optimize performance.  Genomics and scientific organizations are deploying technologies producing ever granular bits of potentially important information.

Documents & Other Unstructured Data

Virtually every organization produces an immense amount of unstructured data, or in other words information that does not easily conform to a defined data model.    This can include internal documentation, publications, correspondences, health records, audio recordings, etc.  Not only is this a content management problem but it also requires unique analytical techniques to harvest value from the content.  Increase the scale of it and it now becomes a Big Data challenge.  Businesses can use this data for ensuring compliance, managing risk and achieving more complete records.

Many more organizations will have “Big Data” challenges over the coming years. Some of this can be attributed to their individual growth as a company but much of it is the result of technology advances and outside factors.   It is safe to conclude that all organizations with Big Data will need to take some action to do something valuable with it, at least to remain competitive.

In future blogs I will talk in more specifics about individual approaches, technologies and business application around Big Data.  In the meantime, please feel free to comment below or reach out to me to talk about “Big Data” challenges you are facing.

After conducting numerous initial assessments for ERP implementations and upgrades, we have seen that all too many reporting solutions were not built according to data warehousing best practices. Because of this, upgrading to a new ERP release can be almost as costly as rewriting it from scratch.  If best practices are not adopted at the outset, not only will this reporting system upgrade cost more than it would otherwise but future upgrades will only become more time consuming and costly.

Data warehousing (and other) best practices are imperative to a successful ERP implementation or upgrade. The system cannot be stagnant, it must be able to adapt and grow with the business intelligence needs of the organization. The following five principles should be observed:

1. Separating the extraction from the load in the ETL to simplify upgrading and minimize data extraction and load time.
2. Using an enterprise data model, not a subject specific data model, to minimize disruption when new functionality is added to the data warehouse.
3. Employing conformed dimensions so that, for example, one is dealing with the same definition of a vendor whether in procurement, accounts payable, or project work.
4. Providing tools to maximize user self service.
5. Building metrics and attributes that follow business processes so that results from several ERP’s can easily be combined to provide users consistent views across the organization.

An enterprise data warehouse that is fed primarily by your ERP system (that can grow to include other data silos) can provide you a centralized platform for your enterprise business intelligence. It becomes a one-stop-shop that provides an up-to-date status of the health of the organization at any moment.

Also, having a modular approach in ETL design will ensure that your system can adapt to all future changes in the source system- saving time, money and frustration in the long run.

There are many data warehousing solutions on the market that can either be built or bought.  One such solution is from Oracle.

In the last installment of this blog series, we will look into the features and benefits you can gain from implementing Oracle Business Intelligence Analytics (OBIA) during your ERP upgrade. Stay tuned…

What we (Emtec) have observed , and the point of this blog is; in most cases (physical, logical and business process contextual) understanding of the data is left to the interpretation of varied groups throughout an organization.  There’s recognition within business units; a project took longer than it should have done and within IT; a number of the efficiencies of the ETL tool or the BI platform were not realized. Frequently, these concerns are elevated to senior management and ‘harsh words are exchanged and blame is assigned’.

In order to turn data into a corporate information asset you must create data governance around how the organization Manages and Consumes data by embarking upon an Information Management Center of Excellence (CoE).  The CoE would address the problem of projects that run over budget,  outlined above, by defining common business rules, which is the combination of BPI (Business Process Improvement) and the data elements supporting the BPI.

We believe that the Information Management CoE is most effective when the group launches with a core group that accommodates the current state of information maturity and evolves over time to reflect a more mature future state of your information management practices.  We believe this can be achieved by understanding where in the maturity model your organization stands and then begin your journey from there.

Let us introduce an Information Management maturity model comprised of the following 4 levels…

1. Initial – When beginning the CoE journey, start with a single subject area associated with a recent Information Management project such as a Data Warehouse implementation.
2. Managed – The 2^nd stage of maturity is to broaden the scope of the CoE to include multiple disciplines/subject areas such as all of Information Delivery including; Business Intelligence, ETL, a Data Quality project or a Master Data initiative, etc.
3. Enhanced – The 3^rd stage of maturity is achieved when the CoE has constituents that participate from both IT, who are the technical data stewards and Business stakeholders who provide business process context as well as data ownership to the CoE.
4. Visionary – The 4^th stage of maturity when all information is seen as a trusted corporate asset. All data is governed throughout its lifecycle from transactional systems… to operational data stores… to data warehouse & data mart… to end-of-life as archived data.

Adopting this maturity model will allow you to develop the necessary controls, policies & procedures to reap the rewards of being an information-driven organization, and to create competitive advantage from your data.

We intend to build on this theme of a CoE maturity model in a Thought Leadership Whitepaper to be published in the next couple of months.  We welcome your comments and feedback, and anyone providing feedback will receive an advance copy of the Whitepaper.

Until next time.

With no shortage of jargon or cleverly named objects in the IT world, many misunderstand what is referred to as a cube, aka Analysis Cube or OLAP Cube. When meeting with clients or speaking at user groups, this tends to be one of the most commonly asked questions. While many have heard the term before, they are unsure of its meaning and purpose.

A cube, defined

Before I begin, most people are comfortable using and viewing information in a spreadsheet.  Relational databases use a similar format (rows and columns) where two or more tables are joined based on a key relationship. My definition begins with a use-case example that most business users should understand involving Sales Orders.

Now, let’s assume that we have a relational database containing sales information. This database contains Sales Order Header data, Sales Order Line Item Details and a Customer Master table. Each of these tables is two dimensional, and each of these tables can be related to one another.

Instead of fields and rows, a cube is designed with a structure of dimensions and measures.  Dimensions hold attributes we know about our data.  Measures hold things we want to measure (sum, count, average, etc.).  Instead of this data being stored in columns and rows, the measures are boiled down to the smallest level defined by the intersection of one or more dimensions.

Phew!  So now you are probably wondering what this means in English?

Sales Order Example:

Dimensions:

• From the Header: Sales Order Number, Date of Sales Order, Salesperson
• From the Line Items: Item Number, Item Description, Product Family
• From the Customer Master: Customer ID, Customer Name, Customer Class, Ship to City, Ship to Zip, Ship to State

Measures:

• From the Header: Count of Sales Orders
• From the Line Items: Count of Line Items, Sum of Quantity, Sum of Extended Amount

The Breakdown

In this example the cube will store the following:

• A count of 1 for each Sales Order in the Sales Order Header table.
• A count of 1 for each Sales Order Line Item
• A count of 1 for the Sum value of the Quantity and Extended Amount for each line item.

The reason that this design is called a cube is that an answer to a question (query) exists at 1 or multiple intersections of this data. For example, how many sales orders did Sales Associate, Bob, have yesterday with a customer ship-to zip code of 90210?

How do I know when I need a cube?

Let’s assume that you are asked how many Sales Orders (count of Sales Order Header) were processed yesterday (Date of Sales Order). Running this query against a relational table would be a very simple exercise of counting the Sales Orders that all have the date of yesterday. After producing this answer, you are asked which salesperson had the most sales yesterday. Again, not a complicated request, but it does involve changing your query and rerunning it. Then your inquisitive co-worker may want to know the number of Sales Orders by Salesperson but also by Customer Class. Even still, this query isn’t difficult but it does involve joining to the Customer Master table in order to derive the Customer Class – you get the picture.

For efficiency – enter your cube. Imagine if you could provide your co-worker with an Excel workbook that would allow them to drag-and-drop any of the dimensions (Salesperson, Date, Customer Class) that are related to Sales Orders on their own live, interactive report.

All of this would be possible without them needing to know a query language or the relationships of tables. Today they might want Sales Orders by Salesperson and Customer Class, but tomorrow they need the Total Extended Amount by Zip Code and calculated average Sales Amount per Sales Order. When using a cube, all of this would be possible (and fast) without requiring intervention from you or a developer writing queries.

Cube Reporting Tools

You can report from or retrieve answers from your cube using a variety of software tools. OLAP cubes have a native query language (for example, Microsoft Analysis Services uses MDX). You can also interact with cubes using popular desktop analysis software like Excel. Most standard reporting tools also have data sources allowing them to report from cubes (such as Microsoft SQL Reporting Services). Lastly and most impressively, cubes can be displayed as dashboards and KPIs on company web portals.

When should I not use a cube?

Even though the cube is meant for speed and efficiency, certain scenarios are better carried out without one.

I recently met with a client who had implemented a development standard stating all business reporting must come from their cubes. While I admired them for embracing cubes and Business Intelligence, this approach caused some serious performance issues when creating some traditional tabular reports.

Cubes are optimized to do quick analysis of data by different dimensions and are not efficient when returning full tabular record sets. Imagine that the relational table of data is a single piece of paper and the cube is the collection of this paper cut into 1,000 little squares. Creating a full tabular record set from a cube database is equivalent to taping all 1,000 little paper squares back together into a full piece of paper. It can be done but shouldn’t be done.

Now that you are armed with a better understanding of the meaning and purpose of using a cube- make sure the nature of any given task is fully understood before you find yourself taping all those little squares back together again.

Oracle’s Usability Labs are the most comprehensive labs devoted to database technology and applications in the enterprise software space. All major Applications, Tools, and Server products are tested at the labs. Approximately 50 new products and upgrades are tested every year. The results are reported to the product development teams to make improvements before products ship. There is a technical staff dedicated to lab operations and maintenance. This resource is open to all within Oracle who need user testing facilities for Oracle products.

My favorite part was when I was selected to be the test subject using a new technology to monitor the movement of my eyes.  I sat in front of a computer and would read a PowerPoint slide with instruction to look for something on the next screens.  There was a black piece of plastic at the bottom of the monitor that recorded all my eye movements!  The first PowerPoint indicated that I needed to download something.  The recording was 100% accurate where my eyes went to first.  Oracle will take all the data and combine it into the most common occurrences and place the specific items where most people were prone to look.  There was even a one way mirror for the team to watch and observe.  It was very cool and scary at the same time.

So, if you ever have a chance to visit Redwood Shores, California- put this on your list of things to do.  I highly recommend it.

In Part 1 of this series, we discussed the time, resources and budget required for ERP upgrades and the questions you should ask before embarking on an upgrade.  In part II, we will dive into the importance of implementing BI best-practices within your ERP upgrades/ implementations.

The Need for Speed

One of the big challenges that the IT team faces with a matured ERP system is that over a period of time, the performance degrades.  Some of the big reasons for this degradation are:

• An increase in the number of users as the ERP system grows.
• An increase in the amount of data in the system.
• An increase in the number of reports being generated.
• An increase in the amount of “live data” being pulled with each report request.

Challenges with ERP Reporting Functionalities

One of the major challenges with ERP performance is that the reporting tools that come with ERP systems always run against real-time data. For example, in PeopleSoft, all cyclical or ad-hoc reports, whether they are nVision, Crystal or PS Queries all run against real-time data. This unnecessary load on the ERP system causes response time to degrade.

Most reports are historical and do not require real-time data and not all users need real-time data for their analysis.  In fact on average only about 2-5 percent of users require real time data about 2-5 percent of the time.

Additional challenges with ERP reporting tools include:

• Reporting tools provide a static view of the data without the ability to drill down to details.
• Multiple report types are created to obtain similar data due to a lack of customization and/or drill down capabilities.
• Data end-users often do not have the training and expertise needed to run these reports or create new ones. As a result, IT is heavily involved in extracting data for business users, eventually becoming a “reporting house”.

Over time an abundance of custom reports are in the system to fulfill the ad-hoc needs, making upgrades to the ERP and its reports system difficult, expensive, time-consuming and risky.

• Lack of cross-functional reporting. Older reporting  were  created when each functional, geographic, or product area could run with minimal interaction with other areas.

Enter Business Intelligence

Adopting some simple business intelligence best-practices can help alleviate ERP reporting & performance problems.

1. Recognizing where and when “live data” is needed, who needs it, and ensuring a true understanding of how the system “needs to work” and “will work” on a daily basis.
2. Create reporting options, which meet both current and anticipated future business needs.

• With greater demand for transparency and accountability from stakeholders and regulators, along with globalization, mergers and acquisitions, there is a more constant need for inter and intra-company collaboration and improved and transparent management processes.
• Executives are being held accountable for business success and failures, and therefore it becomes imperative to ensure operating information is correct and consistent. They must have confidence in their process and, ultimately, in the results that they are reporting.

When conducting initial assessments prior to client ERP implementations or upgrades, we have seen that many reporting solutions were not built according to data warehousing best practices, so upgrading them to a new ERP release can be almost as costly as rewriting them from scratch.

If best practices are not adopted, not only will upgrades to reporting systems cost more than it would otherwise, future upgrades will become more time consuming and costly as well.  Employ BI best practices now to better manage reporting requirements, get true value from your ERP data, reduce upgrade costs, and keep your ERP system running at top notch speed.

Stay tuned for part III of this series where we will discuss how a good data warehouse and ETL design can decrease maintenance costs and increase the upgradeability of the system.

Since I have been spending a lot of time in the air traveling to client sites lately, I will employ some airline vernacular to explain the history and direction of the suite as I see it.

Cruising Altitude

“We have now reached our maximum cruising altitude”.  For most people, it’s a cue…run to the lavatory, rummage through their overhead bag, pull out their gadgets.  For me personally, it means I can finally pry myself from the “sardine” can they call an airline seat, stretch my legs and feel free again… at least until I need to make way for one of those lavatory-seeking passengers.

InfoSphere Information Server has been running at cruising speed for quite a few years now.   Kudos go out to IBM and Ascential Software (prior to the acquisition) for continuous investment and focus on the product suite.  It is assuring to choose a product that has this level of commitment and dedication from the vendor.

If anyone recalls Torrent Systems, I was fortunate to having worked with their Orchestrate parallel processing engine dating back to the mid-90’s, when it was at a beta validation level.  In fact, I worked on implementing the technology on one of the first clients to use it for a large-scale need. About a year after acquiring Torrent in 2001, Ascential integrated the parallel engine into its flagship DataStage product and positioned it to suddenly become a data integration player for organizations of all sizes.

With that leapfrog, DataStage could scale to handle greater and greater volumes of data and compete with the likes of Ab Initio for enterprise grade high-performance processing.   Subsequent versions by Ascential, and then by IBM following the acquisition of Ascential in 2005, added more and more creditability to the roadmap.  The parallel capabilities continue to extend into more and more components including QualityStage, allowing greater scalability at more stages of processing.  Other investments included greater administration and job management capabilities, tighter integration with IBM and other vendor products, solid version control, ability to execute on more operating systems, and end-to-end leveraging of metadata.

I believe this level of progression allows it to be nicely positioned to support the trends and demands of the industry to come.

Ready for Take-off

“Ladies and gentlemen, we are waiting for clearance. We will take off momentarily”.  Let’s face it; they don’t know how close we really are to taking off.  “Momentarily” could mean anywhere from 1 moment to 120 moments!  Yes- I encountered the latter recently, almost wishing for the return of horse-drawn carriages for cross-country travel.  I am pretty sure they didn’t charge baggage fees.

Information Server 8.7, the latest version from IBM, brings a number of advancements that align nicely with industry trends.  As highlighted in the Top Trends post, Big Data and In-Memory Analytics will be the focus of more and more organizations.  Last year saw a significant rise in client interest for this technology, bringing the promise of delivering integration and insight capabilities in a way never previously imagined.  Teradata’s acquisition of Aster Data, IBM’s acquisition of Neteeza, the rise of Greenplum, newer appliances such as SAP Hana, Oracle Exadata & Exalytics and XtremeData dbX prove that vendors see the trend as well and are gearing up for a big uptick in client demand.

The days of the same ole’ forms of business intelligence and analytics delivery are over.  The Information Server suite can now support big data and Hadoop and has optimized integration with Neteeza, SAS and database products.  Despite the desire of vendors to gain the lion’s share of a client’s wallet, I expect that clients will want a best-of-breed class of applications and products.  The result will be a heterogeneous environment with a blend of vendors and products.  I also anticipate that the Information Server will recognize this and continue to build integration flexibility into their products, including extending it to integrate with more vendor appliances.

Another nice capability with the 8.7 version is more comprehensive information governance.  I believe that data quality and governance will be hot topics in the coming year.  Organizations are realizing that all of these powerful analytical and BI tools provide exciting opportunities but aren’t much good without trusted data.  Information Server is nicely positioned to instill quality in the integration process, integrate with master data management initiatives and allow the same consistent integration policies to be extended to consumers such as real-time services.

Bumpy Air

“Folks, please take your seats, we are about to experience some ‘bumpy air’”.  Interpreted…turbulence may be nothing or it could be really bad.  Get ready to buckle up, grab your glass in one hand and the dreaded unmarked “bag” in the other.  I admit it’s good they use the less unnerving words ‘bumpy air’ rather than turbulence.

IBM has already announced formally that, as of April 30, 2012, a number of older versions of data integration products that were previously packaged under the WebSphere family will no longer be supported.  In general, support will end for versions 7.5.0 and 7.5.1 of various products including DataStage, QualityStage, ProfileStage, and MetaStage.  For those organizations still on or near those product versions, it’s decision time.  This post will hopefully provide some considerations for you to make the decision easier.  Last year we encountered many clients struggling with this decision to upgrade.  Some were brave enough and took it on themselves.  For other clients, we created an approach to help them migrate from older versions of Information Server to the latest and greatest but more importantly help them learn how to take advantage of new features.

A caution to those organizations that are not actively investing to enhance their internal data integration capabilities- the “turbulence” is yet to come.  These trends bringing advanced analytical capabilities into your organization are inevitable and if your integration strategy, regardless of the choice of tool, is not in a position to support the potentially complex needs of these initiatives, plan for considerable challenges ahead.  This is likely a good time to assess your data integration environment, and really your overall information platform, and make the necessary strategic and tactical investments now in support of anticipated demand.

With good planning we can all hope for a smooth and pleasant journey ahead, whether it’s technology, travel or otherwise.

In general, the four main objectives of the annual planning process are:

• Set financial targets and metrics for the upcoming year
• Communicate those targets and metrics to the organization
• Determine bonus compensation for the organization (usually the management team)
• Provide insight into future performance (analyze market opportunities and threats)

Most organizations do a fine job setting and communicating financial targets as well as determining bonus compensation.  Where they falter is in providing insight into future performance.  They tend to rely on historical data for future prediction, which is not a good predictor in today’s economic environment.  In fact, if the current trend of market volatility continues, yesterday’s data will be even less valuable in the future.

To better predict future performance, forecasting models that observe BOTH internal and external market drivers are needed to create these financial forecasts.  Developing financial models is hard work and determining the linkages that drive value for an organization takes time.  However, the benefits provide organizations with a competitive advantage to exploit opportunities before their competitors or simply avoid a train wreck.

An alternative to the Annual Planning Process

The annual planning process often takes more than 3 months to complete, which often makes the annual plan financials obsolete before the plan is even completed.

To avoid this, some organizations are turning to a rolling forecast to “wean” themselves off of the annual planning process.  A rolling forecast is typically a financial plan that encompasses more than one year (18 or 24 months) and is completed on a monthly or quarterly basis, derived from financial models.  The use of modeling has evolved as a result of financial organizations not having the time or resources available to create comprehensive financial forecasts within the normal annual planning process.  Due to these time and resource constraints, the development of a more efficient financial project process using financial modes instead of user input was formed.  This forecasting solution could also be used to develop financial projections based on a set of assumptions or scenarios such; as “Make versus Buy”, the start of a new product line, the entry of a competitor, and commodity price increases, etc.

Most organizations still utilize an annual planning process but may want to look at this alternate option.

Upgrades are usually performed to incorporate functionalities offered by a newer version and to mitigate risks of the current system. However, an ERP upgrade requires dedicated development time, resources and budget. They can take months, even years, to complete, and involves technical, functional and business user resources, to attain maximum benefits.

Key areas of focus that must be planned and budgeted for include;

• Ensuring smooth and uninterrupted data transfer
• Development of work processes and
• Report capabilities to provide necessary data for functionality and decision-making

Data Transfer

Data transfer between systems and/or ERP versions is also essential and can be complex .The volume and complexity of data increases with additional ERP modules, such as, Supply Chain Management, Product Lifecycle Management, Resource Management, and Budgeting and Planning, and must be protected during the ERP upgrade process.

Work Processes & Reporting

Specific work functions must exist to ensure reporting continues and business needs are met. Reports which pull data from the ERP may require modification based on data organization (known as, its “schema”). Systems used to populate operational data stores, data warehouses, or data marts, (known as Extraction, Transformation, and Load, or ETL systems), may also have to be modified. The reports generated from these data stores should be altered to ensure a smooth transition. Depending on how reporting data stores are constructed,

If an organization has not upgraded its ERP and is skipping one or more intermediate releases, e.g. migrating from PeopleSoft 8.8 to 9.1 (skipping 8.9 and 9.0), the scale of the upgrade will increase to accommodate both ERP and reporting needs.

Ask These Questions before you Upgrade

ERP upgrade projects offer an excellent opportunity to reevaluate your BI strategies for your ERP system. To ensure maximum efficiency and value is brought to the project, important questions should be asked by the functional and IT teams:

• How do we increase my organization’s agility?
• How can we foster cross-functional collaboration?
• How do we push adoption of best practice KPI’s?
• How do we increase the organization’s resilience?
• How do we provide metrics that span functional and data silos?
• How can we ease the pain for my next upgrade?
• How do we decrease the load on the ERP system?
• How can we lower my ongoing maintenance costs?
• What reporting should be implemented from my ERP vs. my data warehouse?
• Do changes in the vendor’s technology mean we should revisit other parts of the architecture during this upgrade?

An upgrade project is an excellent opportunity to answer these questions and build an infrastructure that not only supports your ERP system but also becomes a foundation for an enterprise wide BI initiative.

Stay tuned, next we will dive into answers to these questions and why it is important to implement BI during your ERP upgrade. Subcribe to the BI Blog RSS feed to be notified when part two becomes available.

Following is a list of the strategic directions we anticipate our Clients to be taking in 2012. We’ve broken the list down by Macro-trends that are industry-wide. In related Blog postings, we’ll comment on Micro-trends because they tend to be more vendor / product specific.

Macro-trends – Information Management, BI & Analytics:  It always takes some time to absorb the impacts of the Macro-trends, and then place them in the context of how & when Clients will choose to adopt the underlying technologies.

In 2012 we anticipate;

1. Business Intelligence & Social Media – Is feedback from social media sites just “noise” or does the data truly reveal buying trends and patterns of behavior? The larger question is, how will organizations mine the data in order to make better decisions?
2. Time to decision – Every large organization is challenged with increasing data volumes, squeezed into a shrinking Load (ETL) window. The envelope for ETL processing continues to be pushed by business demands for more accurate, timely & complete data.
3. Predictive Analytics – Historically, BI has been as good as the underlying data infrastructure at reporting past performance. BI is moving out of this rear-facing view of the organization’s performance, towards a forward-looking (predictive) view.
4. Big Data & In-memory Analytics – The MapReduce functionality permits BI vendors to scan through large datasets in near realtime Analytical & Data Mining applications. E.g. an ERP transaction could invoke an Analytical function for fraud detection, it has many uses across a broad range of verticals. Arguably the next big step forward for the BI vendors.
5. Data Infrastructure – “DW or DM in a box” has been tried before, stumbled badly and is back again in a different format. One of the benefits of improved ERP & BI integration (I’m thinking of SAP & Oracle) brings tighter integration of Industry Data Models, ETL & Master Data.
6. Cloud, SAAS & onDemand BI Services – For small, medium sized business (SMB) Clients, the costs of ownership (TCO) of a BI or Analytics platform continue to decline. All the leading vendors have an onDemand offering, and a Cloud IT infrastructure can dramatically lower operational costs.

The Micro-trends describe the tactical implementation options the leading vendors are able to offer Clients. We’ll comment on these in additional postings that are more vendor / product specific.

We’re interested in your feedback on how these trends resonate with your organization.

With that, we wish you a happy and successful 2012.

Information Management Practice, Leadership team