Two acquisitions this week – Teradata buying eCircle and IBM snapping up Tealeaf – mark out digital marketing analytics as a focus and investment area for the data warehousing and analytic heavyweights. Although both acquisitions have different technology use cases, they do have  commonality as both have a distinctly mobile flavour; underlining the growing importance of mobile as a channel for marketing campaigns and online customer experiences.

Teradata was first out of the block this week when it announced its intention to buy eCircle, a cloud based European provider of digital market messaging. The company, based out of Munich, helps marketers create targeted and personalised digital campaigns through mobile devices, email and social media.

eCircle will sit as part of the Aprimo group (itself a former 2010 acquisition target) and form part of the group’s integrated marketing management offerings as well as being offered as a standalone capability.  This acquisition however is not only about adding additional digital messaging technology capabilities;  eCircle will help bolster Aprimo’s European presence as well as bringing core consulting expertise and experience to the team.

If not to be outdone later the same day IBM reached into its deep coffers to buy Tealeaf – a provider of customer experience analytics software that helps organisations access information about consumer web experiences. Similar to eCircle, Tealeaf will form part of IBM’s smarter commerce initiative and will get integrated into the company’s Enterprise Marketing and Management Group.

With Tealeaf IBM gets some neat technology that’s able to capture and replay digital interactions across website and mobile devices to pinpoint online customer experience problems and use this insight to resolve and optimise the experience. As we outlined in our report Analytics and the Customer Journey building a differentiated customer experience is becoming more of a competitive weapon for many organisations in face of tougher market conditions and an increasingly empowered and discerning customer base.

The digital marketer’s challenge, of course, is to make sure that every customer experience counts – whether this is in the form of a particular marketing communication (such as an offer or coupon) or an online customer interaction (such as an ecommerce transaction) – and in today’s untethered world this increasingly is happening on mobile devices too. It’s an opportunity that both IBM and Teradata, alongside other players such as Oracle, are vying for; and is part of the motivation behind other buys in this space including IBM’s previous acquisitions of Coremetrics, Unica and DemandTec and Teradata’s acquisition of Aster Data.

And it’s not difficult to see why. From a data management and analytics standpoint modern digital marketing platforms need to bring together a heady mix of capabilities including advanced and predictive analytics, the management and storage of Big Data (especially when you start to factor in social media data streams for instance) that can be exploited to support the delivery of marketing communications and online experiences across multiple channels, including of course mobile devices. These are all areas that IBM and Teradata have leadership positions in and are obviously keen to develop.

Although we’ll have to wait until close to hear the detail behind how IBM and Teradata will integrate and leverage their new acquisitions, I’d be really interested to hear what you think. Do let me know.

If you haven’t already heard on Twitter or the blogosphere, IBM yesterday announced its intention to acquire Vivisimo – a data discovery and navigation vendor primarily for unstructured data – for an undisclosed sum. The company has around 140 customers in industries such as government, life sciences, consumer goods and financial services including Airbus, Procter & Gamble, Bupa, and LexisNexis among others.

Not surprisingly given the current market buzz this deal is being framed as a Big Data acquisition and therefore sits within the company’s Information Management division. Vivisimo is a small but nonetheless interesting player in the federated discovery, navigation and analytics space. In other words its software is good at scanning unstructured (as well as structured) data, automating its discovery and then putting it in a format that can be navigated and analysed by users to provide a single view of their disparate information. One particular use case is the Johns Hopkins University Applied Physics Laboratory, who is using Vivisimo to discover, search, and navigate multiple federated data sources to provide its employees with a unified search tool for content, project and expertise information.

These capabilities are enabled through the software’s federated architecture that pushes out the queries, exploration and analysis of data at source in their respective repositories, rather than trying to bring data together in a centralised repository such as a data warehouse. There is of course a solid argument for federating data discovery and search, as it’s not always practical, timely or necessarily to lug large amounts of data across the network and consolidate it for analysis.

Under the hood Vivisimo’s platform includes data source connectors as well capabilities for indexing, text analytics and metadata support (such as tagging and building taxonomies), all of which underpin its application workbench for developing custom discovery applications.  As mentioned in our Outlook 2012 blog post, analytic search is one of the key technologies trends we see in the market, helping more end users gain access to and insight from their data. It’s also behind similar moves by HP and Oracle who in 2011 snapped up Autonomy and Endeca respectively for similar types of capabilities.

This is a smart move by IBM as it gives their customers flexibility in the design of specific Big Data applications. However, the big question on everyone’s lips is not necessarily why did IBM buy Vivisimo but how is it going to integrate yet another technology piece into its (perhaps one would say all encompassing ) Big Data platform. IBM already has text analytic capabilities within BigInsights (its platform for building Hadoop based applications). Not forgetting this the company already has similar search capabilities from OmniFind as part of its Content Analytics product line that currently sits outside of its Big Data technology offerings. To add to this, the company also has the challenge of trying to unify the different development environments that currently sit within its Big Data portfolio.

Despite these overlaps and integration challenges, there are some natural synergies and use cases for integrating the Vivisimo technology with other parts of its Big Data platform. For instance it can be used for the discovery and profiling of content before it’s loaded into the platform; equally it could be used as a single point of access across a range of repositories – not necessarily those only based on Hadoop – thereby helping customers get to grips with more of their Big Data in a faster timeframe by leaving it within its source system. How it will achieve these aims and how fast it can manage the integration remains to be seen. The company expects the deal to close in the second quarter of 2012.

If you take a look at Progress Software’s home page on the web today, you’ll see the following four things highlighted (along with a blog post from CTO John Bates):

• the launch of Responsive Process Management (RPM) 2.2
• the results of a ‘BPM smackdown’ analysis that placed Savvion in a leading spot
• a case study of Apria Healthcare, based on use of Savvion’s BPM technology
• another analysis showing Actional as a leader in its space.

With the new Progress strategy outlined in a press release today, all of those four things will go away.

Progress plans to divest its Actional, Artix, DataXtend, FuseSource, ObjectStore, Orbacus, Orbix, Savvion, Shadow and Sonic product lines. So – if we take this at face value (and that’s all I can do at this stage) that means no more technology to support customers looking to implement BPM, SOA, or MDM. Instead the company will renew its focus on its heritage OpenEdge and DataDirect businesses, as well as redouble its efforts to build and market a cloud-based application platform, and increase its focus on Apama in Capital Markets (you can find out the detail the press  release).

At this stage, I have to make some assumptions. But if Progress means what people normally mean when they say “divest”, I think these divestitures could place the new growth strategy – focused in significant part around OpenEdge – at significant risk. Why? Because part of what was starting to make OpenEdge interesting to the market again was the way in which Progress was expanding the scope of the development concerns that OpenEdge could address.

There are other concerns; not least, the release highlights “Apama Analytics” and talks in other places about Big Data and analytics capabilities; but these don’t exist in any real sense today (at least, not in the sense that most people would use those terms). An intimation is made that OpenEdge will be positioned as a platform for building massive scale web applications that leverage Big Data, but that’s a long way from what OpenEdge is really doing for anyone today.

According to the release Progress expects to divest all these products/businesses by the middle to the end of 2013 – which is, any way you look at it, quite a long time away in the world of sales. I wouldn’t be at all surprised to see the pipelines for all these products to dry up very soon indeed, as prospects focus instead on potential choices with clear futures. I hope that’s accounted for in the plan!

Now of course this is a quick reaction to a significant piece of news for any Progress customer or prospect. Of course I’ll be looking for more in-depth information as soon as I can get it – and if I learn anything which adds to this or contradicts it I’ll update you all. At the moment, though, it looks like Progress is set to do a 180 degree turn, and revisit its past to try and reinvent its future.

Let’s see how it turns out… and hope that some of the very decent technologies being punted here find good homes.

I’d love to get your thoughts and comments!

UPDATE 30/4/12:

I had a briefing with John Bates and Colleen Smith of Progress on 26 April, the day after I wrote the piece above, and I learned some additional detail that I wanted to share with you.

Firstly, Bates and Smith were very clear in saying that the company needs to find a space where it can be truly different for a profitable market segment, and use this to drive growth. It’s focusing on what it calls “Application Platform as a Service” (aPaaS) and at this point, its intention is to get to be the “number 1 provider of application development and deployment platforms in the Cloud”. The current enterprise IT middleware business (combining the business from Savvion, Actional, Sonic, et al) just isn’t providing enough of a growth engine for shareholders; and the company’s leadership also feels that Progress has struggled to focus sufficiently as it’s been fighting on a large number of fronts. By adding data connectivity (DataDirect), event processing (Apama), rules (Corticon) and visualisation (Control Tower) capabilities to the work it’s already done through Progress Arcade in making OpenEdge a foundation for a PaaS offering for ISVs wanting to transition to cloud delivery, Progress believes it can find that space. It’s also planning to add support for multiple development languages, further boost analytics, and create some explicit linkages to “Big Data” management technologies.

My take is that if Progress can execute on the plan it shared with me, then it could do something really interesting – particularly for small and medium sized application vendors. But I also think it’s got a hell of a task ahead of it to get all these additional products – all of which are built for on-premise installation – and re-engineer them sensibly for PaaS. What’s more, because things are moving so fast in the PaaS world, I think its window of opportunity might prove very tight to squeeze through.

Secondly, although Progress plans to divest the product businesses as laid out above to help it focus much more clearly, it’s currently looking at ways it can continue to make use of some of the Savvion technology in combination with OpenEdge – so it can continue to offer what it currently calls OpenEdge BPM (particularly of interest to its ISV partners). I didn’t get a detailed view of what this might involve from a potential engineering and licensing perspective, and I think it’s probably too early to say. However the company is prepared to go on record to say “While [the] intent is to divest the Savvion (BPM) and Sonic (ESB) products, [Progress is] committed to supporting features that are essential for building and deploying agile, next generation applications.”. This is something that anyone interested in the future of OpenEdge needs to watch carefully.

Third, Progress is going to try to find buyers for its divested product lines as soon as possible; it understands that while there’s uncertainty in the market about what will happen next, those businesses could very easily freeze. Its business plan does assume some short term revenue decline as part of this strategy shift, but my view is that nevertheless Progress needs to work very quickly and diligently from here on; the initial communication fell some way short of a well-managed message IMO. Progress is also confident that it will find buyers quickly; my advice to any nervous customers using the to-be-divested products is to keep a very close eye on where those products go. Your strategy could be affected if a key technology you rely on ends up being owned by an outfit primarily concerned with milking maintenance revenues.

A few weeks ago I was fortunate enough to attend a Data Scientist series event in London sponsored by EMC/Greenplum. Its aim was to bring like-minded people together to explore the topic of creating competitive advantage with Big Data analytics. As a very well-attended event it also provided a perfect opportunity to discuss the role of data scientists and similarly how – given the current advanced analytics skills shortage – companies should go about sourcing this next generation of analyst-cum-scientist.

Part of my discussions at the event focused on pinning down a description for a data scientist, a task that proved to be harder than you would think. Overall consensus however seemed to be that they broadly fall into two camps. Firstly, those that represent the purist view, where a data ‘scientist’ is responsible for experimenting, discovering and publishing research on new algorithms and analytic techniques, activities prevalent at academic institutions and data driven organisations such as Google and Yahoo; and second, the data scientists who could be best described as more mainstream – those responsible for experimenting, creating new knowledge and developing insights for real world commercial business problems. My first impressions from the event were that many of the attendees naturally fell into this second category.

So what makes this type of data scientist? To begin with it’s a role that needs to mix both old and new disciplines, tools and techniques and combine this with a healthy dose of analytical scepticism and curiosity!  Similarly it’s a role that needs to blend advanced analytic skills such as those evident in projects for churn modelling, fraud detection, customer segmentation, risk mitigation or sentiment analysis – a set of disciplines that requires a deep knowledge and proficiency in areas such as predictive modelling, text analytics, machine learning and data mining.

In my experience any project involving these more specialised skills requires a person who is comfortable with preparing and exploring data, has an aptitude for applying analytic techniques and algorithms to data and is highly competent at discovering and identifying meaningful patterns in that data. In many cases this person is a statistician or mathematician, someone who is numerically sophisticated and data savvy; but someone who is also (affectionately) known, due to their clinical data analysis approach, as ‘the guy or gal in a white coat’ – a fleeting reference to their laboratory style existence perhaps.

While these traits provide a solid grounding for the data scientist there are other qualities, skills, experience and abilities that need to be brought to bear if a data scientist is to become more than just a re-badged statistician. Above all the data scientist’s role needs to become more critical and central to the end-to-end analytics process, something that can be enabled by;

• Building expertise in handling large volumes and different styles of digital data, such as unstructured and real time data
• Becoming conversant in new analytic technologies, architectures and languages – where necessary – for storing, processing and manipulating this type of data.
• Developing a flair for the exploratory and experimental side of the role; required to tease out interesting and previously unknown insights in vast pools of data.  The best data scientists tend to be both intensely curious and compentent communicators.
• Enhancing their ability to participate in the analytic process. Rather than work in isolation, the success of a data scientist’s analytical prowess also hinges on working in unison with other members of the analytics team whether this is the data engineer, DBAs, programmers or business analysts.
• Bringing a high degree of data domain experience or business know-how to the table so that analytical insights can be effectively applied to real world business problems or opportunities.

It’s perhaps the latter three areas of skill in this list that more clearly differentiates the work of a data scientist over previous incarnations, but in many ways, it also illustrates a more subtle and generational difference between an old and new style of data analysis; where different and modern technologies and languages such as Hadoop, R, Perl and Python are increasingly being put to work on Big Data Analytic problems.

Of course the big question surrounding data scientists is how do you source, train and retain these highly valuable individuals? Given that data science is an evolving discipline it’s no surprise to find that many practitioners often move from adjacent areas such as mathematics, software engineering or computer science for instance. But it’s a problem that is also exacerbated because of the limited amount of graduate and post graduate courses dedicated to advanced analytics and/or data science – although this is changing. Many of those looking for alternative strategies are starting by targeting their efforts internally and investing in home-grown talent. While there’s no silver bullet to the skills shortage problem casting your net further could yield valuable results, whether this through recruiting different types of graduate, internal training or sourcing talent through a third party. And for those of you lucky enough to have these talented individuals already on board then my words of advice would be – hold onto and nurture them and above all don’t let them go!

Although it’s not a formal part of the BPM research programme I set out in advance at the end of last year, in the past few weeks I’ve been drawn into looking in some detail at the emerging process mining space.

Process mining has been an active academic research space for some years (and eminent BPM research leader Prof Wil van der Aalst leads a team at TU/e which has been instrumental in launching commercial projects as well as advancing research in the area).

The Process Mining manifesto, published late last year, is a really good overview of the area and call to action – and one thing it makes clear is that the scope of process mining technology is much broader than process discovery (which is the area that most commentary has focused on so far, with much discussion conflating the two concepts).

According to the manifesto,  there are three applications of process mining – discovery, conformance checking and enhancement:

• Discovery is about taking event logs and analysing them to produce models of work.
• Conformance is about comparing existing models of work with evidence from event logs to discover any operational ‘gaps’ between prescribed or recommended practices and actual work in the field.
• Enhancement is about using data from event logs to enrich the information provided by static models – perhaps by overlaying performance information, for example; or even using historical event information to predict the performance of work currently in progress and suggest ways to optimise it ‘on the fly’.

What’s particularly interesting to me, based on my reading of the manifesto at least, is that the authors (or at least some of them) appear to propose that process mining in its broadest context provides the foundation for a different kind of process management system from the kind many people are familiar with today – one that’s ‘passive’ rather than ‘active’.

This ‘passive system’ is not like today’s BPMSs, which manage processes and the execution of work using those processes through a core co-ordinating application that orchestrates the flow of work between people and systems.

Rather, through ongoing and continuous mining of event logs ‘in the background’, not directly connected to the systems that people use to get work done, such a system would work by detecting the shadows that work casts onto existing IT systems; tracking those shadows in the context of models (discovered or purposely created); and then using that analysis to drive a) management insights into opportunities for improvement and b) operational insights into optimal execution of work.

As the manifesto itself points out, the engineering and research foundations are already in place to make a system like this possible today. Such a system would have the potential to deliver many of the benefits that today’s BPM projects can deliver, but without interposing a new application layer that risks disrupting relationships that people have with their existing working habits and IT systems.

Still, though, I think it’s going to take a few years before such systems gain significant mainstream traction in industry. Why? Because a lot of the practical detail of implementing such a system in industry would require new tools to be built, and the big vendor money is currently being poured into ongoing marketing and improvement of today’s generation of BPMSs; and there are no vendors of any significant size that could release such a platform in the near future without confusing the hell out of its prospect and customer base.

I think we will see systems like this start to be deployed in the coming years, particularly in scenarios where ‘unstructured’ knowledge work is at the heart of the business domain under consideration – but that doesn’t mean process mining is a dead-end: far from it.

I really think we’ll see a lot of real-world deployment of process mining’s discovery application, and quite soon (in the coming months). Why? Because in this context, process mining techniques and technologies help to address an immediate pain point that an established community of industry practitioners have. Specifically, how to quickly discern the actual state of work in a given area of a business to provide a reliable foundation for analysis of improvement opportunities.

I’ll be looking a little more at this ‘discovery’ aspect of process mining in a forthcoming post. In the meantime, I’d love to get your thoughts on this larger question – it’s still an emerging area and I know I am a long way from having all the answers!

Of all the conversations and discussions held at Salesforce’s customer day in Egham this week, one in particular struck a chord with me. Cloudapps, an ISV and partner of Salesforce, spoke briefly about the challenges of capturing, storing and analysing Big Data, in particular telecom mast sensor data, on the Force.com platform.  Although there wasn’t time to drill into detail about Force.com’s Big Data support – it got me thinking generally about the wider options for exploiting vast amounts of digital data within the cloud.

As a lot of the Big Data that people are interested in analysing – such as sensor data, social media data or weblog file data – already lives in the cloud, it makes sense to also use it as a platform for hosting and analysing this data too. Rather than the often more inefficient method of pushing it to an on-premise enterprise data warehouse for example.  At the same time one set of technologies in particular – Hadoop – is also becoming synonymous as a lower cost approach for storing and processing these large-scale datasets in the cloud.

Given the proliferation of digital data and the desire to harness it for better business effect, it’s not surprising that Big Data and the Cloud are on a natural collision course. With its elastic processing capacity, lower cost and lower risk approach, the cloud provides a powerful platform for storing, processing and crunching this data – whether using Hadoop or not. Similarly the emergence of cloud based offerings that mix analytics and Big Data in the public cloud are also helping to circumnavigate some of the skills shortage issues relating to advanced analytics techniques and Big Data technologies.

However that’s not to say that the cloud should be seen as the answer to all your Big Data needs. On-premise data warehouses that employ technologies such as MPP analytic databases, in-memory computing, columnar databases or packaged appliances provide equally valid alternatives. In fact in many cases these Big Data approaches should be seen as complementary to each other, as each brings different strengths to the table.  The challenge however for organisations that haven’t put their eggs in any one particular Big Data basket, is how to mesh these approaches together and equally how to do this across the on-premise and cloud divide.  The benefits of  integrating data to support a more consolidated, complete and accurate view of your business are well known after all.

As we outlined in our recent report on Analytics in the Cloud, a hybrid cloud is one plausible approach to using both public clouds and private or on-premise IT to deliver a more integrated Big Data analytic system. This can provide a more pragmatic and blended approach for balancing the strengths and pitfalls of both cloud and on-premise implementations but it also comes with its own set of challenges. Apart from managing the environment there are also factors relating to the immaturity of certain Big Data technologies, lack of best practice and interoperability across platforms.  For example, of those looking at Big Data Hadoop projects a significant proportion are still in experimentation (rather than production) mode, testing out the concepts, design and technology -although we do expect this to change over the next 12-18 months as the market evolves.

Given the great many opportunities for leveraging Big Data in the cloud, it’s surprising to see that aside from its social media monitoring platform Radian6, Salesforce doesn’t have a stronger message or story about its Big Data hosting capabilities. It appears from the conversations at the customer event the company still has something to prove when it comes to supporting and helping ISVs and developers work with Big Data especially on the Force.com platform. But as a company used to pioneering cloud based offerings we don’t expect this to be the situation for very long.

Today we launched a new online, on-demand event focused on social analytics. As MWD’s Communications Director with responsibility for our online marketing and social media activities, this is a topic close to my heart. So it was with much interest that I listened to what my colleagues Angela Ashenden and Helena Schwenk had to say…

There are four webcasts to listen to, and we’ve purposefully made them short and sweet – long enough to give you some useful information and food for thought, but not so long to put you off even getting started. Time is money, right? Watch them at your leisure, skip about through the presentations using the interactive console and come back anytime you’ve got a few minutes to spare – they’ll be available to watch for a year. I’m confident though that what Angela and Helena have got to share will keep you there once you’ve started listening – it’s interesting stuff. By the way, the webcasts are free to watch, and you don’t need to register.

Click here to visit the event site to start each webcast. For now, here’s an introduction to each one:

1. Do you need social analytics? Here Angela provides an overview of the emerging trend of social analytics, comparing it with similar trends such as social media monitoring, and examining how it can help organisations derive value from its different social channels.

2. Enabling an integrated marketing strategy. In this presentation, Helena examines why marketing has become the focal point for many social analytic efforts and outlines the opportunities and challenges that may arise from its integration across marketing.

3. Alternative scenarios for social analytics. Angela outlines a variety of different contexts in which social analytics may be applied, contrasting public social media scenarios with those of more targeted internal or external social software implementations, and illustrating the role social analytics can play in delivering value from both investment areas.

4. Preparing your social analytics strategy. Helena outlines some vital next steps for any organisation looking to capitalise on its social media investment using social analytics, identifying potential pitfalls to be avoided and challenges which need to be overcome.

I was particularly taken with what Helena talks about in this slide:

I won’t spoil it for you and do a full recap here – you can listen to Helena here – but suffice it to say that this model certainly resonated with me. There’s certainly no shortage of tools available to help me count the impact of what we’re doing as a company… What’s fascinating me at the moment is how to take this to the next level and really make connections between social media and our other marketing activities as a springboard to better understanding our community and striking relationships and engagements with individuals. Sometimes that’s seemed a pretty big chasm to cross, and so far we’ve made some headway using a mix of advanced tools and some good old-fashioned, shirt-sleeves-rolled-up manpower to find our way forward (and for us, compelling content and advice is key of course – no tools can make up for a poor business proposition). But will there ever be a “silver bullet” social analytics engagement tool that will do the job of connecting all the dots at the press of a button? I’m not so sure. But I’ll leave it to Angela and Helena to let me know… If you’re a vendor of such technology, I’m sure you’ll be wanting to let them know.

To finish, I’d be remiss if I didn’t mention that this event is brought to you free of charge thanks to the generous support of our sponsor*: SAS. After you’ve listened to the webcasts, why not check out the SAS ‘exhibitor booth’ page in our virtual exhibition hall? There you’ll find an overview of their social analytics offerings, as well as a wealth of other free resources, including a video and some report downloads.

So get started now and check out the event site. As always, let us know what you think. Use the feedback link on the event site and you’ll receive 10% off a selection of premium advisory reports.

* A note about sponsoring a MWD Advisors event: We’re grateful to our sponsors for their support. But just so you know, we designed and created the event content ourselves – it’s a completely independent and unbiased piece of work. We hope you find it useful. Do let us know.

Just before Christmas, TIBCO released its fourth-quarter and full-year financial results for 2011 (the intervening holiday is why I haven’t blogged about it until now).

While companies like Oracle struggle to deliver performance, TIBCO continues to march relentlessly on. And although it’s perhaps tempting (certainly in comparison to behemoths like Oracle) to paint TIBCO as a small player, its full-year revenue for 2011 was only a step away from the $1bn mark (at $920.2m). That’s 22% growth over 2010′s headline number: an impressive performance.

I mentioned back in early October that TIBCO has begun to extend its business way beyond that of its core enterprise middleware heritage, into new segments that are more line-of-business focused and which enable it to sell much more widely than its financial-services historical heartland. This seems to be continuing apace: a glance at the list of big customer wins sees a much more balanced list with significant contributions from telecoms, government, energy, manufacturing, life sciences and transport/logistics.

TIBCO’s tibbr enterprise social software platform was also called out explicitly in discussion of the results: it seems that this product – built in TIBCO’s labs rather than being acquired – is turning out to be one of the key levers that the company can use to get it into new conversations and relationships. TIBCO’s Spotfire analytics and visualisation technology is also responsible for a big chunk of the growth – as well as helping the company get into more and more deals outside of its historical industry base.

When it comes to BPM technology, ActiveMatrix BPM continues to grow but overall license sales here are a small contributor to TIBCO’s numbers. That’s also currently the case for Nimbus Partners’ business (acquired in August 2011). However BPM technology is actually core to many of TIBCO’s propositions – so over the coming year we’ll be looking for signs that this part of TIBCO’s portfolio is pulling its weight.

It’ll be really interesting to see where the company takes its business in 2012 as moves beyond the magic $1bn to the next step up in size. More acquisitions are on the horizon, I expect – both in core technology and also in industry-specific application and process areas. What do you think?

In our 2011 Outlook we predicted that investment in Analytics and Information Management (AIM) technologies would remain strong despite the challenging economic backdrop as companies looked to sweat their enterprise data to help support key operational and strategic business initiatives. Not surprisingly given the continuing economic uncertainty we expect this trend to continue into 2012, although this time organisations will want to get smarter and more streamlined in deployment, as increasingly cost conscious organisations look to do more and more sophisticated types of analytics and information management with less money and IT staff.

Here’s how we expect things to look over the next twelve months – leave us your comments and let us know what you think. We’ll revisit these in a year’s time and see how things panned out.

Top 5 trends for Analytics and Information Management in 2012

1. Continued move towards analytics as a ‘business as usual’ activity. The days when BI or analytics was a background activity performed by a discreet and elite number of analysts are now outdated. Today organisations are applying analytics and information management techniques across all parts of the organisational landscape to support and underpin critical decisions, whether this involves streamlining business processes, cutting costs, overhauling the customer experience, managing financial governance or reducing exposure to risk. At the same time, this trend is also indicative of a deeper move towards operationalising aspects of the analytic process by enabling more users to access, understand and act upon analytical information — sometimes in an automated and closed loop fashion — in the context of their daily business processes. As part of this move we also expect many organisations to look beyond foundational analytic tools towards more advanced capabilities such as data mining, predictive and scenario modelling, text and sentiment analysis as demand for more accurate and actionable insights continues to grow.

2. The continuing transition towards self-service BI.  The rise in the consumerisation of IT together with the increasing demand for real-time location and context based business information is placing a sharper focus on the ease-of-use, flexibility and the reach of BI and analytic offerings. The move towards end user self-sufficiency will impact the way BI and analytics insights are developed, consumed and delivered. In 2012 we will see a combination of form factors coming to prominence as they help deliver improvements in the user experience including mobile BI, integrated analytical search, in-memory analytics, interactive visualisation and data discovery. This in turn will place additional pressure on IT, who will increasingly be called on to govern, control and manage new and more diverse communities of users but without relinquishing some of the businesses new-found freedom and flexibility.

3. Bringing a social element into corporate decision making. As a continuation of the trend above we also expect social software to have a greater impact on decision-making processes in 2012. As analytics and collaboration capabilities move closer together companies will be forced to look at enterprise collaboration in a more focused way that is tied to key business processes and goals, both within and even across the extended enterprise. By absorbing collaborative styles such as interactivity, information sharing and community-driven networks into the user experience, organisations have the opportunity to bring a degree of control and understanding to the decision-making process that in turn helps improve key processes and makes decisions more consistent, repeatable and transparent.

4. Big Data projects take shape; skills development remains a priority As we highlighted in our 2011 Outlook report, Big Data — specifically the ability to manage and extract value from new, complex, unstructured and voluminous data sources — will continue to remain a challenge and opportunity for many organisations going forward into 2012. As predicted, methods for storing and processing Big Data such as Hadoop and MapReduce have grown in popularity and use in the last year as organisations have started to experiment with Big Data technologies as a complementary or alternative form to traditional data warehousing architectures. In 2012 many of those organisations will move out of experimental mode and start to consider more production status deployments, although the scarcity of skills in Big Data technologies together with the relatively immaturity of tooling will continue to present challenges.

5. Analytics and Information Management in the cloud slowly gains ground. Economic pressure on budgets, together with concerns over skills and the cost and complexity of deployments will mean organisations continue to explore the use of cloud deployments for analytic and information management projects in 2012. Uptake and broader adoption of the cloud as a deployment model, however, will be dependent on business requirements for data latency, data volumes, data security and back-end integration. At the same time we expect to see a significant number of analytic and information management vendors utilising cloud deployment models in 2012 as they look to widen the appeal of their offerings at increasingly attractive price points.

The outlook for 2012

The state of the nation for Analytics and Information Management

We expect demand and growth in the analytics and information management market to remain robust during 2012. Despite the economic backdrop and its impact on IT spending, the prospect for continuing investment in analytic and information management technologies remains good. That’s because, as companies brace themselves for further economic headwinds, many find these technologies provide one of their best lines of defence by helping to catch, retain and focus on their most profitable customers, enhance cross-/up-selling, negotiate more skilfully, improve productivity, reduce service costs, and pinpoint areas of the business for cost-cutting.

As a maturing technology, the use cases of analytics and information management permeate a wide range of industries and sectors. In 2012 we expect this trend to continue, although industries with significant amounts of big data such as retail, telco, biotech, manufacturing, government and Web 2.0 companies will start to branch out in their investments to incorporate newer technologies such as Hadoop and MapReduce.

Key AIM adoption characteristics

For many organisations 2012 will be a time when they look to prioritise their analytics and information management efforts to those initiatives that provide more focused, practical and hard line business benefits. Those projects that successful use analytics to, for example, predict and anticipate market changes before competitors, detect fraudulent transactions, accurately predict demand and boost productivity are likely to garner the most support. Likewise, the heartland of analytics — in particular customer analytics — will continue to be the focus for many projects in 2012, particularly as organisations operating in mature and competitive markets push to improve and strengthen the overall experience they offer customers in a timely and more accurate manner.

Key trends in practice

In 2012 we expect best practices around analytics and information management to evolve in a couple of ways. Firstly, the move towards self-service will inevitably shift the balance of power and ownership of analytic projects and budgets towards business users, giving them more direct control over the information they access and use that in turn leads to better satisfaction and success with BI, analytics and IT in general. Similarly, IT will transition its role from the owners of the analytic environment towards one where they are responsible for oversight, security and governance, thereby freeing them up to work on more value-add activities.

Secondly, as IT budgets continue to be pressurised there will be a renewed emphasis on quicker, lower risk deployments that help deliver a faster return on investment (ROI) and (if possible) under the budgetary radar. This will translate into a solid demand for service providers whose implementation experience, global and regional presence can play a significant role in helping organisations realise the benefits from their analytic solutions.

And finally, uptake of analytic competency centres has been sporadic to date. However, we expect activity to increase during 2012 as companies look to learn from and improve on their early departmental initiatives and move towards a centralised management point for consolidating expert decision making and IT resources around analytics.

Key technology trends

The key analytic and information management technology trends to look out for in 2012 broadly relate to five core categories: being able to harness and driving value out of Big Data; leveraging smart mobile devices and in-memory technologies for delivering more timely, relevant and personalised information to users; utilising the cloud to lower the cost and complexity of deployments; applying the best principles of collaboration within the context of corporate decision making and lowering the barriers for analytic adoption through easier tooling and simplifying the process of embedding intelligence into the operational layer.

Key AIM supplier trends

In the last five years the analytic and information management market has consolidated around a handful of mega-suppliers comprising IBM, SAP Business Objects, Oracle, Microsoft and HP (through its Vertica and Autonomy acquisitions). Despite their dominance, the independent vendors such as Teradata, Microstrategy, SAS and Qliktech continue to grow and remain relevant through their steadfast focus on the core competencies of analytics and information management. Similarly, specialist vendors such as YellowFin, Lyzasoft and Paraccel continue to differentiate themselves through new technology innovations and deployment models. While acquisition activity is expected in 2012, the economic environment may also mean that those with enough cash reserves might be able to pick themselves up a bargain.

Similarly, we expect vendors to continue to focus their efforts on reaching out to business communities; one such way this will manifest itself is by targeting more offerings towards business and industry issues with line-of-business and vertically-aligned applications and services, such as improving the financial close, analysing customer sentiment and real-time fraud detection.

What do you think?

What are your plans for analytics and information management in 2012? Leave your comments here or drop me a line at helena@mwdadvisors.com – it would be great to get your thoughts. To stay in touch with developments throughout 2012 subscribe to my analytics blog feed and for research report updates.

SAP HANA, the company’s in-memory appliance, took top billing and once again dominated the keynote sessions at SAP’s premier business event, SAPPHIRE Now . This year in Madrid the company announced, amongst other enhancements, a version of HANA that runs SAP NetWeaver Business Warehouse (SAP BW), enabling users to tap into the high speed loading and query performance of HANA for near real-time analysis of large data sets and instantaneous insights. This announcement sees SAP evolving and widening the use case for HANA (from its use as a high speed ERP operational data mart environment) at the same time as providing an opportunity to seed its in-memory technology across other parts of its business analytics customer base.

BW is evolving to a fully in-memory enabled data warehousing solution on top of HANA

BW is SAP’s enterprise data warehouse system that has been deployed by over 12,000 customers. It packages together ETL, a data warehouse including a star schema (known as an Infocube), and reporting, planning and analytic capabilities. Although a successful product for SAP the company has for some time now looked towards in-memory technology to help improve the processing and analysis performance of its data warehousing offering. It was therefore only a matter of time before SAP announced a tighter interplay between HANA and BW to enable customers to engage more cost-effectively in faster and near real time planning, reporting and analysis of BW data.

In practice, running on top of HANA means some of BW’s core functions such as the calculation and planning engine, analytic indexes and data store objects can move in memory. Loading, processing and analysing data in memory means faster reporting, shrinking batch windows, the ability to build aggregates on-the-fly as well as removing the need to build some Infocubes, (although not all) which helps reduce DBA maintenance and development effort as well as storage space. However one of the biggest benefits the company is advocating is that this configuration will no longer require a relational database layer such as Oracle or IBM’s DB2 for persisting BW’s Infocubes. By running BW on top of HANA, data traditionally stored in a database can now be moved and stored in an in-memory database. The idea here is not only to make things run lightening fast but also to reduce the layers of infrastructure needed to run your SAP BW environment and thus reduce or at least simplify system administration and operating costs.

SAP has Oracle in its sights

This is clearly a competitive shot at Oracle in particular; the company (much to SAP’s frustration) has a significant number of customers, including BW customers, running on its database. Not only that but Oracle is also advocating Exadata Database Machine as a high performance engine for running SAP applications. By making HANA an attractive option from both a BW performance and cost perspective (through potential database licensing and maintenance savings) SAP hopes to tempt many of these customers to make the switch and migrate to HANA.

That said, SAP’s HANA plans go much further than BW; the longer term aim is to leverage it as a foundational platform for running the company’s existing application portfolio as well as hosting a range of new in-memory enabled apps. In convincing many of its loyal BW customers to move to HANA, SAP is hoping to put them on the right path, so that when – or if  – they choose to run their business applications on HANA, they are in a stronger and more informed position to do so. But at the same time this also represents a significant test for HANA and SAP’s in-memory strategy. If the company is successful in migrating many of BW’s 12,000 customers then it will most certainly pave the way for future development and HANA adoption across other parts of SAP’s portfolio. SAP has invested a lot in HANA and it needs to make it a success.

A smooth migration process is key to success

Part of this success will depend on ensuring a pain-free or smooth transition from running BW on a relational database to HANA – especially when migrating from what is perceived as a reliable database to something that is still relatively unproven may prove to be a significant challenge and risk for some. To allay these concerns SAP has been bullish in its claims of “innovation without disruption” and the relative ease and speed with which customers can migrate to HANA from a traditional BW deployment. We will of course be interested to hear from any customers who can either refute or corroborate these claims. You can reach me at helena@mwdadvisors.com.

Notwithstanding these migration concerns, one of the biggest questions raised about BW on HANA remains its relationship with Business Warehouse Accelerator (BWA). As mentioned previously this isn’t the first time SAP has leveraged in-memory to boost the performance of BW. In fact, BWA can in many ways be seen as a predecessor to HANA on BW in that it loads parts of the application in-memory and leverages BWA indexes for faster response times and performance. Given the obvious overlaps in functionality, being clear about the use cases and future road maps of each product must remain a priority for SAP. When questioned at SAPPHIRE, SAP told us that existing customers can continue to use and invest in BWA but also have the option to move to HANA if they also wish to gain the additional benefits of improved ETL performance and simplified IT infrastructure. Although SAP proposes to enhance and maintain both products, we do believe that over time HANA is likely to supplant BWA as an acceleration tool especially if adoption of BW on HANA proves to be a big success. We looking forward to tracking this story with interest.