Even worse, many (if not most) corporate apps  (and much of the data and documents) probably won’t live in IT’s data centers behind the firewall in the near future, as they progressively move into the cloud. A similar situation is happening with mobile, as apps, driven by the iPhone and Android devices, are embraced by enterprises. Data silos are everywhere, inside and outside the enterprise.

And as the whole BYOD transformation is actively demonstrating, a substantial proportion of user devices are unlikely to come under the control of IT in the future. Forrester estimates, for example, that half of US employees currently pay their entire smartphone service bills themselves, even where they use them to get company work done. And two-thirds of the 20-something workforce chooses its own productivity tools, even when this is against company policy.

The bottom line: businesses of all sizes are going to get more and more complex. And current IT models are probably not good enough to meet the demands of the future.

Your users are now in the box seat – and in just a few years those 20-somethings will be more than half your workforce. Following on quickly behind them will be an even younger generation that’s likely to want to be even freer from IT shackles.

So what does all this mean for IT, a group that exists to combat the kind of chaos we’re describing? How can they maintain the control they need, allied to the security that’s essential to protect the company’s most vital assets — all while delivering the agility to respond to these changes at a cost that enterprises can afford?

Getting your arms around your data and document assets is getting harder and harder. And no company can waste resources on trying to control, secure and manage data, particularly in the current economic downturn world.

This new world requires a new approach to agility. One that can see the entire IT landscape – regardless of structure, format or location – and deliver change in any dimension quickly and cost effectively. IT’s job will continue to be to provide the kind of infrastructure that allows the enterprise to respond quickly to changes in business strategy and seize new business opportunities within meaningful timescales and at appropriate cost.

An interesting recent blog by Forrester’s enterprise architecture analyst, Brian Hopkins, shows though that there is a high degree of skepticism within the corporate world about their ability to deliver the kind of agility that will be needed in the new, excessively dynamic world.

Hopkins says that in Forrester’s “Forrsights Business Decision-Makers Survey, Q4 2011,” nearly three-quarters of business technology leaders rated IT’s ability to establish an architecture that can accommodate changes to business strategy negatively. And over half of their senior IT colleagues agreed with them.  Moreover, only one in five enterprise architects in Forrester’s September 2011 “Global State Of Enterprise Architecture Online Survey” reported their company as being even ‘modestly agile.’

We know we’re in trouble when the majority of the people charged with delivering agility don’t believe they have a good chance of delivering it.  But rather than get totally dispirited, we should pay attention to some more optimistic currents. There’s a new category of software beginning to emerge that deals with the compound complexity of most company’s IT infrastructures, while shielding it from their users.

The new solutions promise to deliver the elastic, user-centric agility enterprises need at a cost they can afford. We call this new approach ‘Content Virtualization’ and there’s been a lot of momentum gathering behind it over the last few months.

Content virtualization is like other kinds of virtualization that have come before: creating an abstraction layer – essentially an index – of all the files, data stores, SharePoint sites and email stores in an organization; then making that index securely available to authorized users.

VirtualWorks is pioneering content virtualization because we see it as the answer to the complexity described above. Overall the goal is to deliver this kind of lightweight abstraction layer, without the need for costly integration work or coding. Speed of deployment and rapid response to change should be watchwords if enterprises are going to be able to deal with the dynamism of their IT environments. These new models have to work without forcing wholesale ‘rip and replace’ of existing resources, applications and data stores.

They shouldn’t force users to change their behavior either, like the big IT infrastructure models have tried in the past.  The goal has to be to give users what they need, while simultaneously giving IT the control and security it must have, regardless of the nature of its legacy apps and data.

If we can deliver that, then IT departments will finally be able to deliver the agility their companies (and users) crave.

Today, for many employees, the sheer volume of data turns it into pure noise. They’re simply unable to process everything coming their way (let alone find the information they really need to make better decisions). It’s got to the point where it’s even been given a clinical name: Information Fatigue Syndrome (IFS).

The result on an individual level is that employees simply tune out all but a small percentage of the available data. So they’ll typically rely most on the information that’s nearby or that they’ve used most often.

On an organizational level, by doing so, your people only get a partial picture on which to make critical business decisions and value is left on the table (or in your data silos).

Indexing fights fatigue
In an ideal world, employees would be able to find exactly what they need at exactly the right time. No matter where it lives or what format it’s stored in, they could simply search and retrieve.

There’s really only one way that can happen: by creating an enterprise index of all information in all data stores, behind the firewall and out in the cloud.

When users need that lost file or elusive chunk of data, they don’t log in to each and every application or database or email server.  They just query the index. Once.

The result: Information Fatigue Syndrome starts to disappear. Because instead of being overwhelmed by data, users gain confidence that they’ll be able to find whatever they need, whenever they need it. And IT folks know that users will only be able to access the information they’re authorized to see. It’s win-win.

With an enterprise index, users won’t have to hoard files – the cause of massive duplication, redundant data and wasted resources.  And Information Fatigue turns into information empowerment.

From hardware costs to bandwidth requirements to user productivity, this data sprawl will have massive effects throughout businesses the world over. Faced with these inescapable facts, the big question of course is: what do you do about it? Here are four things to start thinking about:

1.         Be ruthless about duplicate data
Data multiplies with almost every person it touches. While the world has embraced centralized servers, users are still prone to Save As directly to their hard disks or personal network drives. This means that where you should only have one copy of that 50Mb PowerPoint sales presentation, you end up having a copy for almost every sales person. The result is that your storage requirements mushroom taking your costs with them.

The answer is to track down duplicate files and consolidate them. This in turn can free up significant amounts of space, improving performance and lowering costs.

2.         Put dead data out of its (and your) misery
While you may need to keep a great deal of your data for regulatory and compliance reasons, you do not have to keep everything instantly accessible forever. In the big picture, some data simply isn’t important. Having a robust methodology for when, where and for how long to archive old data is vital. (This is likely to involve a combination of onsite and offsite storage, either physical or cloud-based.)

While it’s an over-simplification, these policies will come down to how you plan to deal with a range of scenarios: gaining instant access to important files, retrieving archived files within an acceptable timeframe, and dealing with disaster recovery/business continuity situations. By being both realistic and aggressive in your archiving, you can significantly reduce the amount of live data you need to manage.

3.         Make data available in one place, where people expect to find it
While data sprawl is a significant IT issue, it shouldn’t be a user one. Spending endless amounts of frustrating time searching for important files is a sure route to lower user satisfaction and a serious drop in productivity. It also results in the recreation of files and yet more duplication (see #1 above).

This will become increasingly important as more and more corporate documents move to the cloud. It will be further compounded by the massive proliferation of devices – laptops, netbooks, PCs, smartphones, tablets – that employees now use to access their files. So it’s more important than ever to give them a single index for all of the files and information they need.

This single index has two important caveats: Neither policy nor elbow grease will drive all data into one place – the index needs to be virtual and automated; the index needs to go beyond file name and right into the data itself, across all of your apps.

4.         Measure the right things
Management guru Peter Drucker once pointed out, “What gets measured gets managed.” This has direct relevance to anyone trying to deal with data sprawl. What you manage will have a major impact on your resulting strategy. For example, if you measure overall system cost-effectiveness, you may go down the route of VDI with indexing turned off. If you focus on employee productivity, however, you won’t sacrifice indexing under any circumstances.

Being clear about what represents success for your business is vital to achieving that success. Understanding the real costs and real benefits is key to determining what to measure and why.

While traditionally the focus for many data management initiatives has been cost (in hard cash terms), today the real need for business is speed. The quicker employees can get the information they need to serve customers, the more competitive the organization becomes. Likewise, how fast executives can make well informed business decisions will affect how competitive the company operates. Measuring your initiatives against this yardstick will ensure that your strategy delivers against the things that matter most to employees, customers and the business.

For a device that was supposed to be just for consumers, the iPad has certainly made an impact in businesses. Apple’s quarterly numbers regularly blow financial analysts’ socks off, and what’s clear is that a surprising part of that success has come from people using iPads at work.

While Apple doesn’t break out the numbers, industry analyst Forrester believes that enterprises spent $6 billion on buying iPads last year. And they expect that to grow by nearly 70% in 2012. Not bad, particularly when Forrester also predicts the (admittedly much more massive) market for Wintel-based PCs and tablets is set to decline by three percent this year.

If the first desktop virtualization wave addressed traditional Windows PCs and laptops (driven by the shift to Windows 7 and the Cloud), the second wave is being driven in large part by the consumerization of IT, which is spurring organizations to embrace new BYOD (bring your own device) policies affecting  - or some might even say afflicting – corporate IT.

For example, a survey of 5,000 customers released last year by Citrix reported that nearly two thirds of respondents planned to use an iPad for business, but only 40 percent expected their company would buy the device. Surprisingly, for those of us brought up in the halcyon days of client-server computing, nearly three-quarters said their companies would let them access corporate data and application resources from their own devices.

iPads have become unstoppable in the enterprise because end-users want the style, ease-of-use, connectivity, portability and downright fun of an iPad. While IT professionals recognize the surge in iPad use, they also want to control and manage these devices in line with corporate IT policy.

And that’s a key element underpinning the second wave of desktop virtualization currently underway. Tablets simply aren’t powerful enough to support traditional client-side apps and data. The result: a surge of businesses are beginning to bring them into their desktop virtualization projects. Both Citrix and VMware report this fact.

IT departments are warming to the concept of supporting iPads and other personal devices (maybe inured is a better description, since a lot of the momentum is coming from company rainmakers like senior salespeople and executives whose voices are often pretty loud). In this argument, desktop virtualization is the catalyst that makes BYOD a reality.

The bad news is that enterprise IT departments are not known as havens of iOS integration expertise. The thought of helping the CEO access Word or Excel  stored on a central virtual environment  with an iPad, changing a file and then re-storing it is enough to send a few shivers down the spine of the most gnarled and hardened systems admin.

The good news is that while IT departments are moving iPad users to Citrix or VMware virtual desktops to handle security and manageability, here at VirtualWorks, we’re seeing customers use our content virtualization solution to turn iPads (and smartphones) into powerful tools in a virtual environment for seamlessly navigating and accessing enterprise information without a keyboard. The initial jumping off point of the end-user experience begins with a simple search bar – and with it, users can access any corporate data or app asset they want easily and quickly.

Our CEO Ed Iacobucci explains how his experience matters to the VirtualWorks challenge.

I thought about this question the other day, and a lame marketing joke came to mind about the difference between commitment and interest. The story compares the relative contributions of a pig and a chicken to a bacon and eggs breakfast: while the chicken might be interested, the pig in question is unavoidably committed.

In this tortured analogy, we find that in many desktop virtualization projects the IT department takes the chicken’s role and end-users assume the pig’s fate.

One big reason may be that IT doesn’t rate users’ needs as highly as the ‘user density per server blade’ ratios necessary to justify their VDI investment. This can be the only reason for the standard practice of switching off native Windows indexing in VDI and published desktop implementations, as recommended by desktop virtualization vendors, consultants and VARs.

But while turning Windows indexing off may get you the densities you need (after all, it is an IOPS killer), it’s a sure fire way to send your VDI or published desktop project downhill. It alienates a lot of users.

The best surprise is no surprise
In the 1950s, the founder of Holiday Inn, Kemmons Wilson, coined a surprising new slogan to attract visitors to his first hotel in Memphis: “the best surprise is no surprise at all.” He believed that consistency was the best path to repeat customer visits. VDI users probably feel the same way.

When users return to work on Monday morning after the desktop virtualization pilot was switched on over the weekend, they want to be able to get hold of all the data they need and the applications they depend on to get their work done. Just like they did last Friday. They don’t want any loss of productivity, control or freedom. And they certainly don’t want a severely degraded user experience.

That’s invariably what they get when native Windows Indexing gets turned off. And the quickest way to a user revolt that threatens to undermine your desktop virtualization plans is to stop your users from doing what they want, and need, to do. Particularly when it’s your company’s best salesperson that can no longer access those sales numbers from his iPad. Or the CFO from finding and using his favorite Excel-based financial model in time for that critical shareholder meeting.

Some IT guys can really hurt themselves on the horns of this punishing dilemma. They simply can’t justify the investment in the project without eradicating the IOPS storm caused by Windows Indexing. One reason indexing is such an IOPS hog is that every virtual machine builds its own index and keeps it constantly updated – it’s actually no wonder that standard practice recommends turning it off. But cutting indexing kills the user experience.

Salvation at hand
Fortunately there’s a solution – one that allows you to switch off indexing and give your users the experience they need.

Content virtualization technologies can be used to solve problems like these easily and quickly. It works by providing end-users with a single, shared high-performance index of all enterprise information within the virtual environment that can be securely searched and accessed from VDI or published desktops – whether the data is structured or unstructured, stored in the cloud or hosted locally.

Content virtualization leaves all your documents and data where they live, but builds a single, sharable index that’s available to every virtual desktop, one that’s updated in real time and does not require heavy lifting, migration of data or special integration tricks. And your security just works.

VirtualWorks has spent the last few years solving these thorny technical issues, so that the users’ and administrators’ experiences are anything but thorny. Our motto: No one should feel like bacon.

The challenges all enterprises face with knowledge management is spawning new tools, new roles and new ideas. At the heart of all of this discussion: How can we turn the rising tide of content into a force that helps us and drives us forward?

Explosive information growth
The amount of documents, emails and other structured information is increasing at an exponential rate. Everyone complains about the trend, but doesn’t do much about it – when was the last time you stopped before adding someone in cc to an email? So, despite initiatives such as email free days the sum total of information stored on company servers is not going to reduce anytime soon. And with new sources such as social media and mobile phone apps coming on stream, the information a company has to manage will continue to rocket in the future.

The multiplication of silos
A direct consequence of this unrestricted growth of information has been the creation of silos of knowledge, isolated from each other, leading to duplication and potential inconsistency. Integrating these and creating data warehouses has spawned a whole new category of software, but still the problem hasn’t been solved. In reality it is unsolvable – while everyone rails against them, silos spring up because they are essentially how humans operate – as Jack Vinson points out we have to put information in buckets that are small enough for our brains to comprehend. Sweep away silos and next day more will spring up – whether it is just keeping documents on a desktop or storing information in emails.

So, given these two factors, is the growing army of knowledge professionals fighting a losing battle when it comes to organising information? Gartner obviously believes not as they see enterprise content management as a key area for business differentiation. M&A in the area, such as HP’s purchase of Autonomy also demonstrates how hot a topic it is in the market today.

A knowledge management shift
What is actually needed is a fundamental change in how we approach Enterprise Information Management. So rather than trying to force people to neatly store information in pre-defined pigeon holes it is time to turn Knowledge Management around and work with human nature rather than against it. Let people save their documents wherever works for them, but put in place an overall, cloud-based platform that has the ability to quickly find knowledge across the entire organisation. This way information professionals can access enterprise critical data and harness it, making sure that knowledge really is power.

Right now, the corporate masters of the universe are struggling with big data – making sense of “the firehose” of structured data. There’s a related – older yet even more formidable – challenge facing every knowledge-based organization: Knowledge management. How can individuals and teams make sense of all the unstructured information at their disposal? Some recent accounts answer: They’re not.

Knowledge Management in the vast ocean of information
Lauri Kivinen is CEO of Yle, the Finnish equivalent to the UK’s BBC in the UK or America’s PBS. This past October, he gave a speech to Finnish technologists at Elcom 2011. In his talk, Mr. Kivinen coined the term the “Anything World.” It’s a world where we are inundated with information:  “Whatever we want, we can fill our personal or professional time with streams of digital services, and we can choose what part of the vast ocean of information and entertainment we want the stream to come from,” Kivinen states. “And we are promised that we can have the content coming to us immediately, anywhere and at any time.”

Sounds great, doesn’t it? But Kivinen went on to describe the dark side of the Anything World: that humans are essentially poor organizers of information and that (left to our own devices) we will rarely find the optimal solution to limitless information. If you need any proof of that, peruse the “filing system” for your digital photos!

Neal Gabler, a senior fellow at the Annenberg Norman Lear Center at the University of Southern California, struck a similar note when he wrote an editorial in the NY Times entitled the Elusive Big Idea. His complaint: that too much information, in particular useless information, was crowding out the good information that helps us make decisions and develop “big ideas.”

While it is easy to relate to information overload in our personal lives, it is perhaps an even worse problem professionally. For example, a 2011 study entitled Overcoming the Insight Deficit by an organization called Corporate Executive Board revealed that many of us struggle to balance data with judgment.  Only 38% of employees – and 50% of senior managers – have the ability to make good decisions based on data. Part of the problem is how information if packaged.  The same study found that 85% of an organization’s information is unstructured, and much of it is unusable.

Intermediation as knowledge management
Maybe the answer is in something Mr. Kivinen said. He indicated that, instead of using tags, folders, cookies and other ways of personalizing and organizing digital information, most people still rely on program directors, editors and other “trusted” intermediaries to help them organize information.

The enterprise equivalent of this trusted intermediary is the knowledge manager. That’s the person or tool that makes it easy for us to find and access the information we need in the haystack of structured and unstructured data that we have to plow through every day.

It turns out that, in the Anything World, we like that.

Do you see an increasing role for enterprise intermediaries in your business?

VirtualWorks’ CEO Ed Iacobucci explains why VirtualWorks is a platform play and what the role of partners is.

Last year, when Gartner asked 220 CEOs for their IT-related priorities for 2012, it uncovered a wide and rather odd discrepancy. CEOs, it transpires, aren’t interested in new technologies.

While their CIO counterparts are champing at the bit to get social, mobile and cloud-based systems bedded into the enterprise, CEOs have their minds fixed completely elsewhere. Total CIO quotes Mark Raskino from Gartner saying:

“Mobile, social, cloud, and the nexus [of the three] — CEOs in midsized to large global companies don’t understand those words.  They do not volunteer terms like ‘cloud’ or ‘social’, and they do not understand how such concepts transform the fortune of their company.”

It turns out that CEOs’ minds are fixed mainly on ERP and CRM – structured, monolithic systems that make the enterprise more efficient and keep all its activities and data regimentally ordered. 

It’s easy to see where the CEOs are coming from: ERP and CRM are huge strategic investments that cost a lot of time and money to implement, so they must be milked for every last drop of value. They’re also vital to the smooth running of the enterprise and to the efficiency of its processes.

But in focusing on these highly structured systems, are CEOs doing their companies a disservice?  Could they be stifling the organisation’s ability to innovate, to find creative new solutions to problems, to develop products and services that change the world?

ERP as the enemy of innovation
It’s a question raised recently by Dave Yarnold, CEO of ServiceMax, who wrote in the Forbes CIO Central blog that ERP systems like SAP are becoming the enemy of business innovation:

“I can’t even guess at the number of meetings I’ve had with senior company leaders over the years where creative new business ideas were shelved because ‘it didn’t fit into SAP’. Is it possible that this long-term adherence to the SAP way has in some way been at the root of the lack of creativity, competitiveness or the loss of manufacturing jobs we now bemoan?”

As CEO of a cloud-based software company, Yarnold has a vested interest in the answer (and as a vendor of content virtualization software, so do we), but the point is a good one.

Studies of innovation have often concluded that too much rigid organisational structure prevents new ideas from forming.  In his 2010 bestseller Where Good Ideas Come From: A Natural History of Innovation, Steven Johnson says that innovation flourishes in a ‘liquid network’:  a loosely-structured environment in which connections can be easily forged and ideas can be easily shared, communicated and accessibly stored.

For Johnson, liquid networks occupy “the fertile zone between too much order and too much anarchy”. He lists cities, the internet and the kind of open-plan offices favoured by digital startups as examples of environments that are fluid enough to allow casual cross-fertilisation of ideas, but not so chaotic that those ideas get lost.

Steering a course between order and anarchy
What do cities and office interiors have to do with CEOs and their technology choices?  ERP and CRM systems are examples of what Johnson calls ‘solid’ networks – rigid structures that don’t easily allow for casual information-sharing, informal collaboration or serendipitous discoveries of other people’s ideas. They may be useful, but they put the brakes on innovation.

On the other hand, the vogue for employees bringing their own smartphones and tablets to work – which we explored in our ‘consumerisation of IT’ blog post – may swing organisations too far the other way; into a world of chaos in which information is squirrelled away all over the place where no one can find or use it.

Johnson says:
“The quickest way to freeze a liquid network is to stuff people into private offices behind closed doors, which is one reason why so many Web-era companies have designed their work environnments around common spaces where casual mingling and interdepartmental chatter happens without any formal planning.”

ERP systems and employee smartphones are both analogous to those “private offices behind closed doors”. They’re environments where information is locked up – either in structured databases or on personal hard drives – and effectively unavailable to the wider organisation.

The smart CIO, if he or she wants to be a catalyst for innovation, must set that information free for the enterprise to use. In doing so they must steer a careful course between what the CEO wants (apparently more rigidity), and what employees want (apparently more chaos).

How they go about doing that is up to them, but we’d like to think that content virtualization has a big role to play. You can find out more about what we mean in our Content Virtualization ebook.

Tell us what you think
What’s your experience? Is your CEO enthusiastic about the potential of mobile, social and cloud technologies – or does he/she think you should be maximising your investment in ERP first?  We’d be very interested to hear your views, so please let us know in the comments.