INFORMation Governance

Implementing an IG Program - Where to Begin?

Elizabeth Kofsky — Wed, 21 Mar 2012 08:15:00 GMT-04:00

Implementing an Information Program - Where to Begin?

I am not sure if this has anything to do with the latest heat wave in Ottawa, Canada – yes, we are experiencing summer-like weather in March. Trust me, I am not complaining, but spring is in the air and it seems like it’s making everyone wake up to a very important topic: where to begin with information governance?

I have received a flurry of inquiries over the past week, all with a common theme - where should our organization start? Something that is critical, and reiterated by many, is to ensure an information governance program lies with the Chief Information Officer and Chief Legal Officer, while involving records management and the key stakeholders from different business units. It is a team approach and not something that can be successful if only driven by one team. As opposed to writing an extensive blog post on all the key considerations, I have decided to talk about just one: start with the riskiest content.

One important element is to look at your riskiest content first as it’s not always possible to conquer all requirements at once. There is nothing wrong with a phased approach; proving success in small increments can prove to be a good approach.

One of the riskiest content pieces is email. Why? Simply because no one ever gets rid of it or if they do, it’s done on a whim with no consideration to its content. This type of activity can translate into an enormous exposure to risk due to the likelihood of smoking guns and spoliation.

So, can email be managed? Absolutley. I strongly encourage flexibility, specifically when it comes to classifying email – one approach will not fit all. There will be some users that want the ability to drag and drop; others will want to choose their classification from picklists or favourites, while others will want the system to automatically classify emails. All methods are encouraged as you want to make sure you provide a system the end-user will use.

As an example, take a look; well actually have a listen to how NuStar Energy addressed its email management challenges.

NuStar Energy L.P. is a publicly traded, limited partnership based in San Antonio, with 8,417 miles of pipeline; 89 terminal and storage facilities that store and distribute crude oil, refined products and specialty liquids; and two asphalt refineries and a fuels refinery with a combined throughput capacity of 118,500 barrels per day.

With operations in the United States, Canada, Mexico, the Netherlands, the United Kingdom and Turkey, the company uses OpenText Email Management for Microsoft Exchange to help reduce the cost and risk of mismanaged corporate email.

“Having all email in one location, being able to search in one place and put a legal hold in one location instead of potentially seven or eight, is huge for us on the legal end. It’s all managed by OpenText."— Clint Wentworth, Records and Information Manager, NuStar Energy.

There has been a lot of traction around the topic of auto classification. A very important element is to make sure the offering is defensible at the same time as being transparent. Defensible so that the organization can state to the courts and auditors the processes taking place to test, tweak and monitor the way the information is being classified. Transparent so there is no disruption to the way our businesses work.

Yes, there is at lot to consider when starting to implement an information governance program. A solid strategy, involvement from key stakeholders and addressing your riskiest content first is a sound way to begin!

Governance: It Doesn't Matter What, It Really Matters Who

Elizabeth Kofsky — Mon, 6 Feb 2012 09:00:00 GMT-04:00

As I just got back from LegalTech NY, where I spent many hours speaking with General Counsels and IT Directors about their business requirements, I wanted to share and repost a blog, which is relevant to those discussions.

The following is a guest post by Dave Martin - @The_D_Martin

Originally posted on Dave Martin's blog on SharePointProMagazine. Read the other posts in this series here.

Over the years one of the many things I’ve been involved with is governance. To most the word governance is synonymous with compliance, which is then in turn synonymous with records management. After that the focus becomes very specific. What I recommend people do when trying to understand how they should approach governance is to approach it as a strategy and make sure that strategy involves and intertwines three things: people, process and technology.

If this sounds familiar it was an integral part the first post I wrote in this series around understanding SharePoint from a big picture perspective. When it comes to governance specifically there is a certain part of this triumvirate that stands out: the people. We often run headstrong into governance deployments without really understanding who needs to be involved before the code hits the servers and processes are under way.

The very first step organizations need to take is defining that small group, who will steer the solution to and through implementation. Obviously IT pops up first as we look to define this working group, and they are unquestionably a very big part as they will be responsible for the technology doing what it needs to do. Another group that should also be considered a bit of a no-brainer is the group or department, or in many cases, the individual responsible for records. This person may be by title the compliance officer, records manager, IT security or legal counsel, regardless they are responsible for the information policy management of the organization. And lastly, but certainly not least we must include someone, or some group that represents the line of business worker, or end-user.

Surprisingly, I have seen this last group consistently excluded from the planning process. Not because they are a problem or difficult to work with, but because the people that are actually going to use the solution are often an afterthought, or as IT would consider them: the customer. DO NOT forget to include this group! At the end of the day they will literally make or break the deployment’s success causing problems for both those other groups at the table as they won’t understand the technology (frustrating IT) or they don’t execute according to policy (putting the company at risk).

Once we have the right contributors at the table we can start to define the governance strategy. When people are defining their governance strategy I always promote that they ask themselves a few key questions to help better understand what they want to do, who it will affect and what they need to do it. Once these questions have been answered a plan can be more easily defined.

The first question is: do you understand your content? This is very important and can also be made as a statement: know your content! We have content broadly spread across our environment, not just in SharePoint. If we are planning to move large portions of that content into SharePoint – file share replacement is one of the top uses of SharePoint – think about what you are moving over. Is this relevant data? Is this data that must live under compliance? Is this duplicate data? Is this active data?

This last question is an important one to consider in terms of SharePoint. SharePoint is an active content solution, and a relatively costly place to store content. If you are moving massive volumes of data into SharePoint it just does not make sense to move old, inactive content into SharePoint from a cost perspective. This content should move directly into an archive that lives on a lower and cheaper tier of storage. Once again we must consider “the who” for a second here. Even though we are moving content out of SharePoint and into a more cost effective compliant place we cannot forget that users should be able to access it or restore it (permissions pending) directly from SharePoint.

My next question is: what are your specific compliance requirements? This varies widely from company to company and industry to industry – every company has corporate policies specific to their internal requirements, and many companies have to adhere to industry regulations. SharePoint does a great job of managing the content in SharePoint as records, but does an even better job when supported by partners. As broad as SharePoint’s records capabilities are when it comes to supporting industry regulations and government guidelines like the Department of Defense 5015.2 (DoD 5015.2), physical records and records living outside of SharePoint’s native repositories a third-party add-on solution is a requirement.

And for my last question, we go back to “the who” again: How will we govern the people? Again, for most, information governance has to do with the information, but we must also be sure to govern the people if we are going to be successful. This question relates to how we are enabling people to leverage the core strengths of SharePoint, and this all starts with the creation of Sites and filling them with content. Organizations have to have a Site provisioning plan in place or they risk putting the organization as a whole at risk. Site sprawl is not just a myth, it is a reality, but it doesn’t have to be feared. Attaching a lifecycle and policies to a Site at the point of creation will ensure that Sites are connected to the data center and can be managed under the watchful eye of IT. Not only this, but we can now monitor those same sites and move them to the appropriate tier of storage once they have become dormant or inactive. Site provisioning allows organizations to permit the creation of as many or as few sites required all in a controlled fashion.

As you can see, understanding “the who” when defining your governance strategy for SharePoint is a pretty big deal. Not to downplay the value of process or technology, but to use an analogy: it is the person that drives the car down the right road, and it really helps when that person knows where they’re going. Just like a good governance plan for SharePoint, people who drive cars will get to their destination faster if they have good maps.

To find out more, join me on February 21st at noon EST where I’ll be participating in the webinar Extending SharePoint Across Your Information Infrastructure. You’ll learn key concepts required to turn SharePoint into a multifaceted, stable, and powerful IT tool set.

Dave Martin is Director of the Microsoft Solutions Group for OpenText. Email Dave at dmartin@opentext.com or follow him on twitter @The_D_Martin

LegalTech NY - 2012

Elizabeth Kofsky — Thu, 26 Jan 2012 08:00:00 GMT-04:00

If you’re like most law firms and departments, you likely have some pretty major concerns with how to manage the explosive growth of information and enormous costs and risks associated with unmanaged information. Sure, there is a cost to store all that information but it’s a small price to pay when compared to the legal and compliance risk and litigation costs that accompany the growth of that content.

If you’re looking for a bit more balance in your firm or department when it comes to creating, retaining and deleting your information, be sure to visit us at LegalTech next week in New York. We’ll help you assess the risk, value and cost associated with your information governance requirements.

Hope to see you in New York

LegalTech
January 30 - February 1, 2012
The Hilton New York
1335 Avenue of Americas
New York, NY

Booth #2205

Argument # 2 for Auto-Classification - Legacy Content

Stephen Ludlow — Mon, 12 Dec 2011 05:20:00 GMT-04:00

In the first part of this series, we looked at the impact of the Nov. 28 MEMORANDUM FOR THE HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES that gives government agencies four months to come up with a plan to improve records management by moving to electronic records management systems - and the argument as to why the Government needs to consider Auto-Classification.

This second part, we look at what should be the easiest argument to make for Auto-Classification, legacy content.

Legacy content is the poster-child for Auto-Classification. If Auto-Classification should be used for instances where humans or business process cannot easily classify content, legacy content epitomizes this. Legacy content is that vast, and growing volume of unstructured information that we have stored in various forms throughout the organization. Legacy content has typically been created by the good intentions of IT to support lines of business and to meet efficiency requirements. Unfortunately, it was also stored before the idea of retention and disposition was on anybody's radar. The most typical scenarios that we hear are:

We have 150 Million emails in a legacy email archiving application
We have 50 TBs of content on fileservers
We have 5000 back-up tapes that we want to get rid of

Throw Away Legacy Content

Organizations are now looking at these legacy stores of information as a source of on-going cost, a source of potential cost should they be forced to discover the content, and a source of litigation risk where information that could have been legitimately deleted is still hanging around.

The problem is, this is not a simple process. In order to "throw something away" organizations need to ensure that the content is not subject to a litigation hold and ensure that the content is not subject to retention policies associated to compliance or internal policy requirements. This means evaluating information to determine where it fits in the RM schedule, and it also means evaluating if it pertains to a litigation hold. In a recent statement to the Civil Rules Advisory Committee, Microsoft stated that two-thirds of its 14,805 litigation holds pertained to matters that had not yet been filed. That is an astounding statement on a couple of levels, but the sheer number of litigation holds is staggering, although not likely atypical. Combine this with the RM programs that often involve hundreds of RM schedules, is it any wonder organizations are not accepting the risk of throwing anything away??

No Easy Button, but…

There is a solution for this conundrum. As usual, it is not just technology, but people and process too. A great starting point for what is typically being called Content Remediation is The Sedona Conference Commentary on Inactive Information Sources. The commentary provides straight-forward advice on the steps that can be taken to evaluate if information can be dispositioned. In the end, the process boils down to a risk vs. cost exercise, where the organization must justify decisions on content dispositioning based on their understanding of the information. A good guiding quote is that, "Generally, organizations will not need to physically review the entire contents of an inactive information store in order to make reasonable, good faith and defensible decisions as to whether it contains information that the organization may be required to retain."

This is where Auto-Classification will be critical. Rather than having to look at every document, Auto-Classification can provide the ability to associate content with the enterprise records schedule, and at the same time, use sampling and review to demonstrate good faith and defensible decisions. Auto-Classification extends the subject matter expertise of the Records Manager into huge volumes of information through the use of analytics and supported by testing and sampling. Obviously, there is a lot of work to be done on the process side, including documenting the steps taken, the risk assessed and the decisions made. Content remediation, supported by Auto-Classification will still not be easy, but the potential savings can be massive.

If the US Federal Government is serious about addressing their electronic records, they also need to be serious about the huge volume of legacy content that they will need to address. Auto-Classification needs to be part of the solution. As with any organization we talk to, the critical question is always, "Will this be any easier next year?"

Argument # 3 for Auto-Classification - The Cloud made me do it will follow soon.

US Federal Government & the Argument for Auto-Classification

Stephen Ludlow — Thu, 8 Dec 2011 09:06:00 GMT-04:00

There was an interesting confluence of events last week.

On Nov. 28, President Obama released a MEMORANDUM FOR THE HEADS OF EXECUTIVE DEPARTMENTS AND AGENCIES that gives government agencies four months to come up with a plan to improve records management by moving to electronic records management systems "where feasible."
On Nov. 30, we announced the availability of OpenText Auto-Classification, the first Auto-Classification solution for Records Management with built-in transparency and defensibility.

We did not plan for these announcements to coincide, but it was definitely good timing. Beyond the obvious additional focus on Records Management the Obama memo has created, a significant argument could be made that the Federal Government will have to adopt Auto-Classification of content in order to be successful in its Records Management programs.In fact, there are at least three arguments:

Argument # 1 for Auto-Classification - We can’t afford not to

Argument # 2 for Auto-Classification - Legacy Content

Argument # 3 for Auto-Classification - The Cloud made me do it

Argument # 1 - We can’t afford not to

The memorandum says, "When records are well-managed, agencies can use them to assess the impact of programs, to reduce redundant efforts, to save money, and to share knowledge within and across their organizations. In these ways, proper records management is backbone of open Government."

So, with all of these obvious benefits, why is the memo needed at all?

Why aren’t federal agencies addressing Records Management on their own without prompting from the Whitehouse and NARA? The truth is, the US Federal government has been largely unsuccessful in managing electronic records, even though more than 90% of all records are now created electronically. In fact, about 95% of federal agencies fail to meet the statutory requirements for maintaining their records according to a NARA self-assessment survey.

Federal Government Agencies are finding out what many organizations struggling with litigation and compliance requirements have known for some time. Records Management is hard! Especially when the scope is expanded to include the plethora of content creating applications and document types that are being used today.

The survey notes a number of issues in the electronic records management programs in in many agencies and noted that these agencies:

Do not ensure that e-mail records are preserved in a recordkeeping system;
Do not monitor staff compliance with e-mail preservation policies on a regular basis;
Have policies that instruct employees to print and file e-mail messages;
Consider system backups a preservation strategy for electronic records, not distinguishing between saving and preserving electronic records;
Consider compliance monitoring to be the responsibility of IT staff; and
Are rarely or not at all involved with, or are excluded from altogether, the design, development, and implementation of new electronic systems.

In this list of issues, we see a number of indicators as to why Records Management is hard in US Federal Agencies. We see is that it is difficult to declare email as records, there are out-dated policies and no enforcement and a lack of archiving. It also indicates that Records Managers lack the ability to impact the selection of applications to ensure that they have Records Management, or Record Declaration capabilities. (BTW - Government Records Managers shouldn’t feel too bad as Forrester’s Brian Hill points out, this is pretty consistent with what is going on in private sector too.)

Many of these issues are a result of the difficulty in rolling Records Management out to the knowledge workers that are creating the records. These knowledge workers require:

Applications where records can be declared and/or captured
Training on how to classify records and the associated change management
Monitoring and enforcement

The reason why Auto-Classification will be critical to making Records Management affordable is that the costs associated to training, change management, monitoring and enforcement will be at least an order of magnitude larger than the cost of the technology.

Auto-Classification addresses many of these costs by taking the knowledge worker out of the equation. Auto-Classification evaluates electronic content as it is captured, and using analytics and rules, can associate an RM classification. Auto-Classification reduces the requirement for training and change management, and most importantly, adds consistency and the ability to monitor and enforce records classification that is just not possible when we depend on knowledge workers to classify content.

Auto-Classification also addresses the single biggest fear associated with Records Management programs, which is the fear of failure. Having end users declare records, especially with high-volume, ad-hoc content like email requires significant change management. This fear of failure is one of the main reasons why many Records Management programs never get off the ground. Records Managers can use OpenText Auto-Classification, in combination with our leading RM and information lifecycle applications, to develop a programmatic, transparent and defensible approach to classifying content that does not transfer the burden of records management to knowledge workers.

Later this week, I will address

Argument # 2 for Auto-Classification - Legacy Content

Argument # 3 for Auto-Classification - The Cloud made me do it

CMIS and Moreq2010: Good Match Or Just Overlap?

Jens Huebel — Tue, 26 Jul 2011 05:05:00 GMT-04:00

Overview

CMIS deals with accessing content in repositories for Enterprise Content Management and given that it tries to cover a broad spectrum of different kinds of repositories. One of the design goals was the capability of exposing existing functionality without the need for changing the underlying implementation. CMIS has built-in flexibility and mechanisms to discover the set of available features. You can even implement CMIS on top of a traditional file system. MoReq2010 on the other side is much more specific in scope. MoReq2010 deals with managing records and this has a bunch of implications on a repository. To perform records management tasks you will need functions that are not available in every CMIS implementation. For example CMIS 1.0 does not know about holds, classification or disposal schedules. On the other side MoReq2010 restricts certain operations or forbids them completely (update or delete operations for example). But still they have some design principles in common: Both standards define a set of services, sets of metadata and type definitions. Both standards also rely on XML for formal data representations. They differ in that CMIS defines two protocol bindings for SOAP and AtomPub as wire protocol. But there is no counterpart in MoReq2010. As primary mechanism for data exchange MoReq2010 defines interfaces and a data format for exporting and importing data (including mass transfers). Specific interfaces for export/import are not covered in CMIS. Let us first look at some more details and derive some options for potential use cases to get synergies from both standards.

Comparing the services

The second part [Part 2] of my blog series gave an introduction about the services in MoReq2010. Refer to this for an introductory explanation. Here we will start comparing what is available in CMIS, what is available in MoReq2010 and how it matches.


MoReq2010	CMIS
User Group Services
Model Role Service	ACL Service / Policy Service
Classification Service
Record Service	Navigation / Object / MultiFiling Service
Model Metadata Service	Repository Service
Disposal Scheduling Service
Disposal Holding Service
Searching / Reporting Service	Discovery Service
Exporting Service
	Versioning Service
	Relationship Service

There is no counterpart for the UserGroupService in CMIS. User Management was considered to be out of scope for CMIS 1.0, because this is a complex topic on its own and there exist other standards for user management. Obviously this leads to some gaps. There is currently a discussion in the CMIS TC around this, so it may change in a future version of the spec [1]. MoReq2010 has specific requirements like unique ids for users and restrictions how to preserve information after removing users.

The ModelRoleService deals with permissions and defines a concrete permission model based on Access Control Lists (ACLs). CMIS has two concepts for permissions: PolicyService and ACLService. The ACL Service is designed to be open for different implementations and only defines some basic rights. The MoReq2010 model is dynamic in the sense that an administrator can define roles (and assign a set of callable methods in that role). The MoReq2010 permission model could be exposed via CMIS. However the dynamic nature of Moreq210 roles is a bit tricky and may lead to confusion for some CMIS clients.

The Classification service, Disposal Scheduling Service and Disposal Holding Service cover specific requirements for records management and do not have a counterpart in CMIS. However not every use case will require the full functionality of these services. Some of the more common use cases can also be covered via CMIS (see below).

The ModelMetadataService deals with metadata and property definitions which are Type Definitions in CMIS. In CMIS 1.0 each object has exactly one type. Moreq2010 is more flexible and also has a concept for templates. The CMIS Repository Service allows only read access to the types, but type creation is on the roadmap [2]. Another proposed extension called "Secondary Types" will give more flexibility in CMIS [3]. This would allow exposing even more of the MoReq2010 model. It would be possible and useful to have common and standardized type definitions for the MoReq2010 system metadata. Not all property definition fields can be exposed via CMIS (e.g. RetainOnDestruction, PresentationOrder, …) but this should not be a blocker and could be covered through CMIS extensions.

The SearchingReportingServices of MoReq2010 is covered by the CMIS Discovery Services, but both have different functionalities. MoReq2010 does not define a query language, CMIS does. Reporting capabilities are not covered in CMIS but could be built on top of the existing service. CMIS also does not have saved searches, but a specific object type might be used as a replacement. It would be interesting to build a MoReq2010 compliant implementation of the SearchingReportingService that relies only on the CMISDiscoveryService for the implementation. In this way one implementation could cover a wider set of existing repositories. This an area where open source implementations would be interesting.

Export and import is a similar story. There is no direct mapping in CMIS for this, but you can implement these services based on CMIS services. MoReq2010 has a lot of requirements on the completeness of data (event history, users, types, etc.). You would need a set of standardized MoReq2010 types in CMIS and it would be interesting to investigate how far you can get providing a common implementation. Probably it will be hard to get full compliance for any existing repository but also a sample implementation as template could be helpful.
For the CMIS RelationshipService there is no counterpart in MoReq2010. However there are concepts in MoReq2010 that can be modeled very well in CMIS using the RelationshipService. An example is the event history. The event history of an object could be modeled as an "EventHistory" relationship type. Having standardized types in CMIS would give us generic functionality in multiple repositories.
MoReq2010 does not support versioning. There are surprisingly little requirements in the specification about content handling in general. The notion of content parts is not supported in CMIS. CMIS only has one content element, but the standard is designed for more flexibility in future extensions.

Overlap or perfect match?

While records management standards covers all the functionality needed for a full blown records system there are many use cases requiring only a small subset of this functionality. Sometimes scenarios in an enterprise require interaction between multiple systems and involve records, but are restricted to certain aspects of a record.
A common example might be an ERP system producing records like invoices, personnel files, financial reports etc. The customer of such a system is interested in managing these records in a compliant way. The ERP system has the business context but is not necessarily itself also a records management system. In this case the ERP system has all the information to assign a MoReq2010 classification ("Invoices are kept for 7 years and use classification INV7") and can pass the invoice as a PDF file with metadata and classification to a MoReq2010 compliant repository. The ERP in this case never might be interested in applying a hold or scheduling a disposition. All the records functionality is exposed from the records system and the business system provides the context.
For use cases like this both standards can complement each other. CMIS is already available for many repositories and applications and has the advantage of providing a binding. Using the binding you simply can plug both systems together. Another system might just need read access to the document (let's say a call center where customers ask questions about their invoices). The list of available classifications might not change that frequently and can be exported and imported (or transferred as another CMIS type).

If a company has more systems managing records in place then they have to be kept in sync to guarantee the company's record policy. Here are many more aspects of a record are involved and the MoReq2010 standard addresses them much better than a generic standard like CMIS can. Such an application for example might want to list all holds, display pending disposal schedules or generate a report with statistics (all covered in MoReq2010).
CMIS may also help in another scenario as an intermediate step if a customer wants to start with classifying today but adds the record management system later. These are only a few examples but there are many use cases where CMIS can be used as a vehicle between applications and repositories where MoReq2010 guarantees the compliance.

Conclusion

There are overlaps between the two standards but this is not harmful. Instead it can help solving use cases based on existing CMIS implementations or taking benefit from the bindings. MoReq2010 then deals with the advanced records management scenarios requiring more knowledge about records management and guaranteeing compliance of the underlying back-end. Many ideas of these articles should be considered being initial ideas inspiring more thoughts and perhaps some objections as well. Nothing of this is written in stone. It will require more investigation and sometimes a deeper understanding of the spec. But I am sure that we will see a lot more interesting articles and use cases within the next weeks and months. Let's make the best out of the two standards by focusing on their strengths where it makes sense.

[Part 1] MoReq2010 And The New RM Debate http://blogs.opentext.com/vca/blog/1.11.623/article/1.26.933/2011/7/7
[Part 2] MoReq2010 A Look Into The Specification http://blogs.opentext.com/vca/blog/1.11.623/article/1.26.935/2011/7/7

[1] User Management in CMIS: http://tools.oasis-open.org/issues/browse/CMIS-724
[2] CMIS Type Mutability: http://tools.oasis-open.org/issues/browse/CMIS-699
[3] CMIS Secondary types proposal: http://tools.oasis-open.org/issues/browse/CMIS-713

Shaping the Connection between RM Principles & Info Gov

Elizabeth Kofsky — Mon, 11 Jul 2011 11:45:00 GMT-04:00

Shaping the Connection between Records Management Principles and Information Governance

I have spent the last couple of weeks speaking with customers about their recent implementations. Due to my background and role at OpenText , the customers I engage with always have a Records Management component in their project. What I found remarkable (and am very happy that it’s actually materializing) is the real shift in the organizational culture surrounding Records Management and Information Governance. It won’t be a surprise to anyone that Records Management used to be seen as people in the basement managing dusty boxes – recently, it has found a new, more vital (no pun intended) role within the organization. I believe a key reason for this shift stems from the fact that organizations are realizing they must evolve and embrace Information Governance (of which RM, Archiving, and eDiscovery are key components). Not only are they embracing Information Governance, organizations are realizing the costs and risks that they will incur if they don’t embrace it.

I just read Barclay Blair’s latest executive brief entitled Justifying Investments in Information Governance which specifically calls out the sources of the costs faced by organizations should they fail to invest in Information Governance. This further substantiates the value of an Information Governance Program.

From recent customer calls, to my delight, I am seeing organizations take the Generally Accepted Records Management Principles (GARP) and apply them to ALL corporate information (meaning, not restricting the fundamental principles of Records Management to “official records” [which we all know only account for a small percentage of all enterprise information]). Looking at two of the eight principles–retention and disposition–I trust you will see the immediate value it brings to all your corporate information.

The principle of retention is about keeping information for as long as required, based on legal, regulatory, fiscal, operational, and historical requirements. This principle needs to be applied to all information so that your organization is mitigating all risks and minimizing all costs associated with information. How can it help? Just look at your shared drives—how many copies/duplicates are residing in your shared drives? How much storage is that taking up? How much does that information increase the risks and costs should your organization be involved in a lawsuit?

You can then add in the principle of disposition, which is about getting rid of information that is no longer required to be maintained by organizational policies or applicable laws. Being able to defensibly disposition information will minimize the risks and costs associated with all of the information. An easy example to demonstrate the value of disposition is to look at an inbox. How much information within the average inbox is not relevant to the business? How many copies of the same attachment does one normally have? How much information could have been deleted years ago?

Looking at these two principles and examples is just the tip of the iceberg. There is a lot more value for your organization by extending all the Records Management Principles to all your corporate information. There is no doubt in my mind, this approach will result in drastic improvements, on how you view and deal with information. I recognize that this will be a transition and will take time, but strongly encourage your organization to start the journey!

MoReq2010 A Look Into The Specification

Jens Huebel — Thu, 7 Jul 2011 03:29:00 GMT-04:00

The first part of this blog series gave an introduction to the new debate about RM initiated by the release of the MoReq2010 specification. This part will be a bit more technical and give an overview about the spec, the ideas and how they were realized. A blog like this can only scratch the surface of such a complex spec. For a full understanding you will have to read the spec, perhaps not even all the 520 pages but some parts: Section 1.4 is an excellent introduction. Each service chapter starts with an overview about the purpose and scope followed by a more detailed list of requirements. The function definitions and data type definitions coming then are more reference material.

And the spec is really quite readable. There are lots of explanations; they start with explaining concepts, ideas and illustrating diagrams before going into the details. In this regard the committee has done an excellent job. One of the most discussed aspects of Moreq2010 is the attempt to achieve better modularity. How does this work and what is different in MoReq2010 compared to similar specifications?

The MoReq2010 core (and this is the only part available yet) specifies a set of services and methods each compliant repository must implement. Services are the foundation how different modules can exchange records or use functionality in other modules. This is the path to go away from having one monolithic application doing everything in a central repository. An example would be an "in-place RM" setup where the records system is outside of the business system. The business creates and stores the data and a separate system does the classification and disposal of those records.

To share data between systems MoReq2010 defines an XML schema and a mandatory requirement for a system to be able to export data according to this schema. The standard has detailed requirements which data have to be exported guaranteeing that all aspects of a record are preserved. For example this includes the history (events) and the user identification. The import capability is an optional feature. Keep in mind that for long retention periods like 75 years, multiple export/import cycles due to system changes are very likely.

Some services usually will exist only once in a deployment and are shared between the different modules. Others may occur in multiple incarnations. User management for example in many cases will be a shared service. Classification on the other side may be implemented in different schemas each one being a service.
The following sections give an overview about the services MoReq2010 defines including their key characteristics and functionality. For a complete description please refer to the specification.

1. User-Group-Service:

Manages users and groups, often may be done within or with the help of the corporate directory, but enforcing certain restrictions:

Users and groups must have a unique id.
Unique ids are never destroyed even if the user/group no longer exists.
Any time even after delete it must be reproducible what groups existed and which user was in which group (provide a report for any historical point in time).
Users and groups have a defined set of mandatory metadata

2. Model-Role-Service

Provides the means to manage access control in a repository. Roles contain a list of functions that can be executed and are assigned to user or groups in Access Control Lists (ACLs). ACLs are attached to records or can be inherited from a parent-child relationship (an aggregation classification). Implementation is optional, but if not available it must be proven that an equally powerful mechanism exists. Roles are divided in administrative and non-administrative roles that have different inheritance behavior. ACLs can include or exclude inherited roles. A System of Records must be able to tell what functions a specific user can use.

3. Classification Service

Classifications are used to associate a business context to a record. They are used to determine the disposal schedule. Every record has a class and only one class. A repository may support different classification schemes and then each scheme will get its own service. Classifications can be inherited. The informational model contains a more detailed description of a hierarchical classification scheme.

4. Record Service

The Record Service manages aggregation of records (in many repositories known as folders). A specialty in MoReq2010 is that an aggregation either contains other aggregations or records but not both. Aggregations are used to manage inheritance of metadata, ACLs and classifications. The maximum number of nesting levels of aggregations can be limited and an aggregation can be closed (means cannot add more entities). Records can be duplicated in multiple aggregations and a record can have sub-components called parts (for example an HTML page plus pictures). A repository must support moving records from one aggregation to another.

5. Model Metadata Service

The Model Metadata Service deals with the definition of metadata sets for records. Again implementation is optional but must be replaced with equivalent functionality if not implemented). MoReq2010 differentiates between system metadata (part of the standard and always mandatory) and context metadata (customer specific). Metadata definitions are grouped into templates and types. Each type has a unique type identifier and each entity in a system has exactly one type. Metadata definitions can have additional constraints like (optional, mandatory, or a maximum number of allowed values). They can also indicate if their values must be preserved when an entity is destroyed. Datatypes indicate if they are textual and have a language identifier. The standard does not define specific data types (like Date, String, Integer, …) but refers to the XML specification instead.

6. Disposal Scheduling Service

As the name indicates, this service is responsible for managing dispositions of records. MoReq2010 distinguishes between deletion and destruction. A disposal process never erases a record completely, but instead destroys it. Destruction means that certain information is still preserved (for example that the record existed, its id, when it was destructed and why, etc.). This is called a residual object then. When retention periods expire, disposal runs are scheduled and these trigger disposal actions. An action must not necessarily be a destroy operation it also can be a transfer operation or some kind of approval workflow. Retention periods have a start date and an end date. Start dates are triggered by event. Such a trigger event exists in many flavors. It can be a metadata change, a change in the parent aggregation, a certain date and so on. Each record always has only one disposal schedule, but this may change over time. An aggregation is destroyed when the last active record it contains is destroyed. A repository must send alerts when disposal actions are not carried out.

7. Disposal Holding Service

Holds are a well known concept in records management. A hold is a legal or administrative order to interrupt the disposal process in MoReq2010. A hold always must explicitly be removed ("lifted"). The Disposal Holding Service contains the methods to manage holds.

8. Searching-Reporting Service

Not hard to guess to guess this service defines the methods for reporting and searching. Searching can be done by metadata or (optionally) as text search in the content. The standard defines a lot of criteria for search and an interesting option to search all metadata text fields simultaneously.

search by entity type, events, any(!) system or context metadata
find records by metadata values
combinations to do complex searches

Searches must reflect access control and search results must be paginated and returned in an ordered list. The standard does not contain a syntax for a query language.
A repository must be able to generate reports (detailed or summary) but makes no assumption about the output format.

9. Export Service

The export service is used to transfer data out of a system into an XML file (or a set of files). Reasons for exporting data can be transfer, migration or replication of data. The XML schema for the data format is not yet specified. Some use cases may prefer or require a partial export of records but this is not further specified and is optional. The service guarantees to preserve sufficient level of detail when moving data between systems (for example includes all events of a record and referred entities like users). Importing data is an optional feature. The service is not only used to transfer records, but also other entities like classes, users, roles, metadata definitions and templates.

Conclusion

MoReq2010 is definitely more than just another records management standard. The service based approach; the focus on modularity and the premise "keep it simple" are all new ways of specifying requirements for records management. But it is also obvious that it is not the silver bullet for everything. There are parts in the standard that will make assumptions about elements being deep in the core of a repository. If a specific vendor model does not fit into this approach it will be hard to implement conformance without breaking compatibility. Other features are not trivial to implement if they have to scale to millions and billions of records in really large systems.

There is also some caution needed to not make expectations that are unrealistic to achieve in the current state. MoReq2010 defines an abstract model with services and methods. It does not define any mechanism how to implement the services or how to access these methods. If this is a Java interface, a web service or something completely different is left open. This means there won't be a test script that you can run against any repository for validation. There won't be a way that you can simply plug different modules together and they can talk to each other. MoReq2010 is not like CMIS and I wonder if this is in scope for a future extension. It does not even define concrete list of input and output parameters for the methods. You won't get a generic client that can browse the pending disposal schedules, list the holds or do a federated query in multiple repositories. This looks a bit like having stopped on half of their way. The XML schema is also not yet available meaning that you can't even begin an implementation at the moment. I fear that this will lead to a lot of frustration when people start looking closer at the standard. But of course this does not mean that there is something fundamentally wrong with it. Many things can be specified on top or added later, but you have to know what you will get at the moment. The committee did an ambitious step. The specification has triggered a new debate about Records Management. Do not underestimate what has happened in the first few weeks after release.
And there are other standards that can fill the gap partially. Have you ever looked at CMIS? Do some of the concepts and terms from above sound familiar to you? Types, folders, ACLs? Is it conflict or a perfect match? A non-relevant replication of things defined elsewhere? I think at least a topic interesting enough to think about in another article.

What else can we do? There is one thing coming into my mind that really would help getting adoption. RM systems are a usually a big investment. It would be helpful to have an implementation to play with. To test, to see how it behaves to check, if and how such a model can be integrated to your system. Ideally something that you can adapt and change according to your needs. Not for production use but for everything else. Something that fills the gaps of the standard, not as a replacement, but as something that turns it into a runnable piece of code, provides a test system and the necessary tools. What am I talking about? Yes, an open source implementation. Ideally it would not come from a vendor but from a neutral organization. Illusion? May be, but also not impossible. There is a community already in ECM, perhaps MoReq2010 will get some attraction…

Part I: MoReq2010 And The New RM Debate

Part III: CMIS and Moreq2010: Good Match Or Just Overlap? http://blogs.opentext.com/vca/blog/1.11.623/article/1.26.954/2011/7/26

MoReq2010 And The New RM Debate

Jens Huebel — Thu, 7 Jul 2011 02:30:00 GMT-04:00

Recently a new version of the MoReq specification (MoReq2010) has been published [1] by the DLM forum.
MoReq2010 is different in many aspects from other standards in the records management area including previous versions of MoReq itself. This has led to an interesting debate about a new era in records management [2], [3] [4] [5].
Let's take a look at the specification: What it is about and how it is different to other records management standards?

Records Management is about managing the lifecycle of corporate records from the creation at the beginning, the flow through business processes, the preservation and archiving phase and finally the disposal. Records Management standards for document management systems have the purpose to guarantee a certain behavior of the system that allows enforcing rules how to manage your content. Those rules can have their origin in legal requirements, company or department policies. There are many aspects of such a system behavior, just to name a few:

prevent deletion before an assigned retention period has passed
guarantee access only to authorized users
keep track of changes and an audit trail for each record
reporting and query capabilities
classification of records making them manageable on large scale
following workflows and processes for disposition of records

RM standards are often similar on a high level but they differ when it comes to details or they focus on a certain application area like US DoD 5015.2 for the military government.
Previous attempts to standardize and certify repository behavior are often specified in text form. An example from the DoD spec 5015.2 version 3:
"RMAs shall provide the capability to define different groups of users with different access privileges. RMAs shall control access to file plan components, record folders, and records based on group membership as well as user account information."
Test cases then are concrete scenarios and look like this: Jan Rangel and Dan Martinez have access to all record categories and record folders. Users assigned the Local Records Administrator Role have permission to create folders in those categories to which they have filing access.

Over time you can imagine that demands and requirements have increased. The rule set gets more and more complex. The test cases focus on one specific use case which often is very different from those of customers in their daily business. More complexity leads to more complex implementations to more specializations, to longer specs taking more time to get finalized. And of course more complexity always means more risk for inconsistencies or implementation issues. Sometimes this has weird implications like features only implemented to pass certification or configuration switches always turned off except for certification.
Customers still often insist on certification according to specific standards. Not because it reflects their business needs but they want to get an approval from a neutral instance that the system follows records management rules. Not perfect for them but provides peace of mind and assurances for compliances.

Most often howeverorganizations exist in a world different than what the standards anticipate. RM standards traditionally define a system living in a closed world. There is one system managing all records and a records manager overseeing everything and having all under his/her control. Reality is much more dynamic. Organizations change and with that their structure and responsibilities. There are mergers and acquisitions bringing new RM and non-RMrepositories to manage. Any larger organization will have more than one document management system in place. There may be some specialized for the needs of a specific department. Others may be replaced or updated with newer incompatible versions. Vendors may change due to shifted priorities in their product lines. Some systems may be discontinued and disappear from the market. Very large organization must decentralize somehow to keep their systems manageable. All this heavily influences the organization of your records, but RM standards do not address these areas. Sometimes not even a full-blown RM system is required; put perhaps there is just a need to get a little bit of structure in your 100TB file shares without moving them entirely.

MoReq2010 is the first attempt to go away from such a centralistic and monolithic behavior. MoReq2010 tries to reduce the functionality to the required minimum. Instead of global behavior it defines a set of services, functions and data types plus their behavior. Some of them are mandatory others are optional. Later more specific modules with additional services or data types may be added (for example a module specialized for healthcare requirements). MoReq2010 does not insist to have everything in a single system. Classifications can be handled in a different system than where the content is stored. MoReq2010 defines how to export and optionally import data to get them from one system into another and to guarantee completeness and consistency. It does not assume necessarily that you have a single file plan for the entire company but it addresses how to share one between multiple systems. Services can be tested in an automated way. There will be no debate how to model a certification test case in your system and whether the UI is appropriate for that task. Instead you can run a piece of code performing method calls and checking conditions and finally indicating success or failure. You can distribute responsibilities between system calling methods and agreeing on data structures and formats.

So as a second example here is another excerpt from MoReq2010. In addition to a behavioral description MoReq2010 defines a service dealing with user management and a function to create a user. Compare this with the example from above:

3. User and Group Service
F14.5.179 User – Create
System Identifier: 2cde7448-6c71-4cff-988a-973e0701a824
Title: User – Create
Description: Create a user
Entity Type: User (E14.2.16)
Entity metadata modified:
• System Identifier (M14.4.100)
• Created Timestamp (M14.4.9)
…
Purpose: Access control , Event generation
The idea is: Keep it simple where possible. Focus on key areas and do not try to address every aspect in one system. Make data shareable and exchangeable but ensure consistency and well defined behavior.

Now the big question is: Is MoReq2010 the revolution in RM? Is this a new era or just another nice try of a theoretical attempt? And how does it work? Let's have a closer look at the spec. Well I guess this is another article…
Will they be successful? This is difficult to answer and the specification is just yet released. Customers will decide if they have addressed the important areas. Vendors will decide if their ideas are implementable with a reasonable effort. All this will take time and like with every other standard it is the chicken-and-egg problem at the beginning… There are many factors making a standard successful or not and not only technical ones. But the debate that has started is promising.

We would love to hear some comments from you: What are your expectations from the next generation records management system? How could a standard help you? What are your main pain points today?

And you don't want to read through the 520 pages of the spec? Then wait for the next part:

Part II: MoReq2010: A Look Into The Specification

Part III: CMIS and Moreq2010: Good Match Or Just Overlap? http://blogs.opentext.com/vca/blog/1.11.623/article/1.26.954/2011/7/26

[1] http://moreq2010.eu/
[2] http://thinkingrecords.co.uk/2011/05/06/how-moreq-2010-differs-from-previous-electronic-records-management-erm-system-specifications/
[3] http://www.realstorygroup.com/Blog/2162-Moreq2010-a-DOD5015-slayer
[4] MoReq2010 : Met with Enthusiasm and some Criticism
http://blogs.opentext.com/vca/blog/1.11.623/article/1.26.855/2011/6/7
[5] http://ecmtalk.libsyn.com/ecm-talk-007-the-launch-of-mo-req-2010

MoReq2010 : Met with Enthusiasm and some Criticism

Tracy Caughell — Tue, 7 Jun 2011 06:00:00 GMT-04:00

MoReq2010 – met with Enthusiasm and some Criticism.

The Modular Requirements for Records Systems, MoReq2010 ,Volume 1 was published June 6, 2011. It is an achievement accomplished from herculean efforts of its author Jon Garde, but also, in no small part, a volunteer effort that has been met with both enthusiasm and criticism.

Enthusiasm of the underlying concepts - the need to have standard structures in place for managing active content and content to be preserved properly for all time in all systems; criticism of ‘do we really need another standard?’, or ‘don’t we have a standard for that already?’.
Enthusiasm for the concept of crowd-sourcing the review; criticism of not allowing enough time for review.
Enthusiasm for the potential for feedback to be incorporated into the finished product; criticism for not including all comments.
Enthusiasm for the idea of evolving the standard over time and allowing for versions and ‘pluggable’ modules; criticism for the confusion that will bring to a certification effort and tracking which versions of which modules and which products go with each.

I could go on and on about the highs and lows throughout this process, but one thing is clear – experts, RM Professionals, vendors, consultants and analysts around the world have taken notice. MoReq2010 is hot off the press, and one thing is clear – it will not go quietly into the night. Even the criticisms are welcome as this will help shape the evolution of the standard, and we are all welcome to take part in the evolution – get in the game, follow the DLM Forum, attend the sixth triennial conference in December 2011, sign up for the upcoming workshops, and keep the conversations going on Twitter with #MoReq2010.

More about my personal experience in this process later, but for now I am off to read the spec with great anticipation – did my comments make it in? how large were the changes? Perhaps a few highs and lows and enthusiasm and criticism are in my near future.

Tracy

Getting Rid of all the stuff... is that reality?

Elizabeth Kofsky — Wed, 1 Jun 2011 02:26:00 GMT-04:00

Getting rid of all the stuff.. is that reality?

I just spent the last week attending the Managing Electronic Records (MER) conference and listening to some fabulous sessions. I was pleasantly surprised the see the focus of many sessions on addressing the challenges that organizations are facing today. Yes, some speakers focused on topics that organizations will not get to for a while, but overall, the quality and content of the sessions was relevant and excellent. In addition, I spent many, many hours (actually days) speaking with prospects and customers about their Information Governance programs. One theme, or topic of discussion, kept emerging–dispositioning content; or should I say, “How do we get people to destroy content?”! As you all know, our inherent behavior is to hoard all our information because, you know, we will need it someday… right? Not!

The fact is, keeping everything forever poses way too much risk and, not to mention, costs way too much. Yes, you may keep hearing storage is cheap but it’s really not so cheap once you add in all the costs associated to reviewing content in response to an eDiscovery request. So how can organizations move forward and get rid of the information they don’t need? They need to start by understanding the value of an Information Governance Program and how Records Management is just a component of it. Yes, an important one, but nevertheless, just a part of it.

Records and Information Management (RIM) is the field of management responsible for the efficient and systematic control of the creation, receipt, maintenance, use, and disposition of records, including processes for capturing and maintaining evidence of and information about business activities and transactions in the form of records. Why can’t we leverage what we know from a RIM perspective and apply it to all information? That way, all the key stakeholders in your organization that care about the costs, risks, and value of information will benefit. IT will have less information to store, Legal will have less content to sift through, the End Users will be able to leverage up-to-date information that is pertinent to their role and business, and RIM will extend their role to more than just records by owning the policies that will dictate when all information can be deleted! A win-win for all, don’t you think?

I do realize it will take time, money and lots of education, but it can happen with a realizable ROI.
So where do you start and how do you begin to make this happen? It was clear from many of my conversations at MER that RIM, Legal, IT and End Users are starting to make this happen within their organizations – an example is the movement towards a broader classification model (also referred to as the big bucket approach). This in itself helps simplify the management of all the information out there. Second, I suggest reading Barclay Blair’s latest Executive Brief: Information Governance & Records Management: Understanding the Difference. It will provide some specific steps to start to make this happen.

So, is getting rid of all the stuff a reality? Absolutely!

Is MER still relevant?

Stephen Ludlow — Sun, 22 May 2011 05:16:00 GMT-04:00

Yes!! And err.... No. And perhaps maybe.

I'm headed to MER today. I think this will be my fourth or fifth year in a row. Let's face it, MER (the National Conference on Managing Electronic Records) is a different kind of conference, for both exhibitors and for attendees. For attendees, MER offers three days of excellent speakers presenting on RM issues, plus the opportunity to interact with a small number of vendors that are specialists in the RM space. For vendors, MER offers a unique way to interact with customers and potential customers. We schedule a series of 30 minute meetings where attendees can ask us just about anything. The 30 minute format gives both the attendee and the vendor the opportunity to get into more detail than is ever possible on the tradeshow floor that is typical to other conferences. I personally enjoy the opportunity to interact with so many different organizations with their broad variety of processes and issues in such a short period of time. Of course, both attendees and exhibitors take full advantage of the opportunity to network and socialize with peers.

So, if the format is effective, is the content still relevant. Absolutely. A quick scan of the agenda shows sessions on topics that are top of mind for organizations facing issues driven by the glut of electronic information, a tightening regulatory environment and the omnipresent impact of litigation and eDiscovery. There are sessions on taking RM practices and applying it to all of the information in organization to gain control through retention and defensible disposition. There are sessions on how opposing counsel might try to take advantage of an organization's RM and information lifecycle policies and practices. There are sessions on the impact of moving applications to the cloud, sessions on the impact of social media, and sessions that discuss the pros and cons of managing records in place. All of these topics are things that need to be taken into consideration when it comes to an RM program and Information Governance at large.

With an effective format, and relevant content, what is stopping me from saying that MER is relevant. If you ask me, the single greatest indicator that it might not be is just down the street. At the Fairmont Chicago, blocks away from MER, Inside Counsel is running their 11th annual Super Conference. That means that while Records Managers are discussing the impact of information governance practices on their organization, their GC's, AGC's and paralegals are discussion the exact same things in a completely different forum. Don't get me wrong, the IC Super Conference covers a lot more ground than just litigation readiness and eDiscovery, but a quick look at the sponsoring exhibitors certainly demonstrates that it is one of the key topics for the conference.

If you ask me, this is just a small example of the largest challenge that we will need to overcome when it comes to addressing Information Governance issues. I think it might best be summarized "preaching to the converted" syndrome. Records managers get together with other records managers. Lawyers get together with other lawyers. IT professionals get together other IT professionals. (Strangely, there does not appear to be too many conferences for end-users - the people most likely impacted by decisions made by these other groups - long-live the water cooler) The groups that are impacted by the challenges associated to eDiscovery and Information Governance are learning from their peers, but they are not learning what is collectively affecting them. This of course, is the challenge for Information Governance, balancing the needs of legal, IT, RIM, and the business - or said differently, ensuring that risk, cost and productivity are well balanced.

The challenge for MER, and the Records Managers who attend MER, is to concede that it has to be about more than just RM. Records Managers are influencers in the policies, practices and technology that manage the lifecycle of information, but they are not the ultimate decisions makers anymore. The best thing that Records Managers could do is invite their boss in compliance, invite the person in IT who is responsible for archiving and storage, and invite their AGC responsible for litigation. MER has to keep offering the same great content, but in order to continue to be relevant, they need to address the larger issues of information governance. This will mean expanding the audience who is invited and ensuring that the content also meets their needs.

Tomorrow is a new day. The agenda looks great. Maybe this transition to a wider audience has already happened. I'm looking forward to meeting new organizations and catching up with old friends.

As an aside, if you are attending MER, remember to drop by the OpenText booth. We will be giving away a short booklet that is a composite of essays by Barclay Blair and Tom Jenkins on information governance.

Information Governance and eDiscovery in Canada

Stephen Ludlow — Mon, 9 May 2011 10:06:00 GMT-04:00

Living in Ottawa, Canada, but specialising in Information Governance and eDiscovery has meant a lot of stamps on my passport. One of the last times I crossed the border, the US Customs officer quipped that I was in the States more than he was. That is why I am very happy to be speaking locally for a change. AIIM is coming to Ottawa, and for its inaugural session in Ottawa, it will tackle eDiscovery Issues and Trends that Impact Canadian Organizations. I think that there is some significance to the fact that eDiscovery is the first topic that will be tackled by AIIM in Ottawa. In the United States, eDiscovery has probably been the single biggest impetus for organizations to look at their information management practices and take on Information Governance as a means to control the risks and costs of eDiscovery.

I believe that eDiscovery will have the same transformational effect in Canada as well - albeit in a more low-key, pragmatic (some might say Canadian) way. There have not been block-buster cases in Canada that everybody can point to as a warning, nor eDiscovery rock-star judges like Judge Scheindlin, Judge Grimm, or Judge Peck, but eDiscovery in Canada is both gaining momentum and is a growing concern. The provincial superior courts are adopting Sedona Canada principles and there is a growing body of eDiscovery precedents. There has also been a up-tick of firms offering eDiscovery services.

Another reason that it will be interesting to present in Ottawa is the obvious connection to the Federal Government of Canada. The Government of Canada is one of OpenText's largest customers, principally using our Document and Records management solutions. The Government of Canada is not a stranger to eDiscovery and certainly not to Access to Information requests, but to date it has not significantly adopted cost saving measure based on retention and disposition practices of getting rid of transitory content. It will be an interesting panel discussion when we start talking about the benefits of not saving EVERYTHING.

Finally, I am quite pleased to be on a panel with Dominic Jaar from KPMG. Dominic has worked with ARMA, the EDRM and Sedona and brings a wealth of experience to the panel - particularly from a Canadian perspective. I'm looking forward to his insight.

Hope to see you at the Ottawa Public Library from 9:00 - 11:30 on Thurs. May 12th. You can register online.

Your Information Infrastructure...Part III

Elizabeth Kofsky — Tue, 3 May 2011 01:30:00 GMT-04:00

Your Information Infrastructure... More than Just SharePoint - Part III

As a new blogger, I have officially executed on my first “faux pas du blog” and I do have to say, there is no guarantee this will be my last. I stated a couple of weeks ago that I was going to discuss SharePoint Governance in a two part series… well, I was mistaken as it has now gone beyond that, but who ever heard of a two part series anyway? Through this additional post I hope you will get some additional insights and value as this series continues.

The series so far has focused on an organizations information infrastructure – specifically, that SharePoint, although significant, is only part of it. The first post focused on the considerations one must make when deploying SharePoint sites and the second discussed some additional considerations in terms of governing the information itself.

I recently had the pleasure of discussing some of these very issues with many energy, financial and government customers at our Calgary Content Day. What clearly resonated with the audience was the understanding that Information Governance is evolving within organizations – that organizations realize they need more efficient ways to connect business critical information in a manner that maintains the end user experience while striking the balance between the risk, cost and value in the creation and storage of information. More importantly, it became clear to me that organizations are embracing and deploying technologies, specifically software solutions, to enable them to achieve good information governance.

In this part of the series, I would like to take the considerations that have been previously discussed about good information governance, in addition to the considerations that Barclay T. Blair has noted in his latest Executive Brief on SharePoint Governance:

· Can your information requirements be met with SharePoint alone?

· Do you have the right corporate governance structure?

And discuss some of the key elements needed to achieve the ideal solution.

Here are some of the key elements that must be part of the ideal solution.

1. Supporting Centralized Governance. Ensuring that all your organizational content, including SharePoint, Email, File Shares, ERP, etc. is managed centrally under a unified set of policies. These policies will ensure the organization and all of its information is being managed against regulatory compliance and internal corporate policy requirements.

2. Controlling Site Deployment. Making sure that SharePoint site creation and deployment is being managed and aligned to policy through automated, structured and controlled processes with built in compliance. This ensures all sites will be created in a structured way, connected to the corporate data center and retired when they have lived out there value within SharePoint.

3. Preserving Business Critical Content. Supporting the ability to archive and restore SharePoint documents and sites will reduce the amount of inactive content sitting idle in SharePoint servers while optimizing performance. Reducing storage requirements through de-duplication, single instance archiving (SIA) and compression, and enabling compliance to industry regulations and corporate policies by defining a long term preservation strategy.

4. Maintaining End User Experience. Ensuring the end user experience is never affected and making sure they have access to all content through SharePoint, even if that content is not natively stored within SharePoint.

These elements will ultimately benefit the organization as a whole as it allows:

· end users to create, access and share relevant, up-to-date information in their user interface of choice,

· preserves information that is required for legal and regulatory compliance, and

· ensures that out of date and duplicate information is managed and eliminated (once all regulatory obligations have been met of course),

all of which, reduce the risks and costs associated with your information, while at the same time, leveraging its value.

You may be asking, has the ideal solution ever been implemented? Without this being a promotional piece for OpenText... YES!! Many organizations are successfully implementing Information Governance solutions for SharePoint.

Your Information Infrastructure...Part II

Elizabeth Kofsky — Mon, 18 Apr 2011 05:00:00 GMT-04:00

Your Information Infrastructure... More than Just SharePoint - Part II

In my previous post I mentioned that we have to really start thinking of SharePoint as part of the information infrastructure, not as the information infrastructure and then discussed how we need to govern the solution – managing the SharePoint deployment and how it will ultimately be used. In this post I want to target what we should be considering when we’re governing the information itself.

So, what should you be considering? The technologies (archiving, retention and records management, eDiscovery…), the stakeholders (IT, Legal, Compliance, RIM , the end user…), the value, the risk and the costs associated to creating and storing information. In Barclay Blair’s most recent executive brief, he made a few assertions about the role of the CIO in information governance.

Assertion #1: Do you have the right tools?

Sure, less of an assertion and more specifically a question, but it made me think back to what we had said in the previous blog that SharePoint must be well integrated into your existing information infrastructure. If we want to ensure good information governance, ideally we want to have centralized management of all of our information solutions (Email, file systems, and ERP to name a few) and more importantly we’ll want a unified policy structure to make sure all of the information is being managed according to corporate policy and industry regulations. To do this, you’ll need the right tools.

Let’s take th example of archived information for a second. Even though we have numerous content repositories spread disparately across our organization, it’s not all tied into active processes and a good portion of it may be business critical and need to be accessible, managed, or retained, for long periods - many industries such as life sciences, oil and gas, utilities, government and financial services have a high level of regulatory requirements. And again, although we need to make sure we are implementing good governance practices to SharePoint we need to consider how all of those other repositories of content will be managed, retained and defensibly deleted under one roof and a single set of rules. A big added value of the centralized management of various content sources is our ability to now provide litigation readiness as we are retaining all content according to a unified set of policies and ensuring that it is defensibly deleted. This and the fact that all of the business critical content is now in one place, makes discovery a lot more efficient.

But information governance isn’t just about the information, good information governance actually has a lot to do with the infrastructure it lives on. Now I say this due to the impact of another quote from our friend Barclay:

Assertion #2: …the title Chief Infrastructure Officer is perhaps more apt.

I couldn’t agree more, but I see why people think Chief Information Officer works too; the CIO is after all in charge of the infrastructure that holds the information, so by default he is responsible for its well-being. With all this said, information governance for SharePoint can really be seen as a huge value add to the CIO, because it is not only going to reduce headaches around his ownership of the content, but also help drive improved efficiencies around the systems that hold it all.

How so?

Well we’ve already talked about the compliance and centralized management benefits of archival but what about the storage management benefits? On average 25% or more of an organization’s SharePoint sites can be inactive, which means we can move a quarter of that content from costly production servers to much cheaper storage for long-term preservation. Now storage management benefits don’t end with old content, as with SharePoint there is an ability to redirect active objects (referred to by Microsoft as Binary Large Objects or BLOBs) themselves into a more cost effective device, all the while ensuring transparent access of this content to end-users. This process for active content is commonly referred to as externalization.

Ah yes, the end-users… the bane of every CIO’s existence. For the end-user it is all about transparent access, as they are ultimately the CIO’s only customer. The good news is, those storage management/optimization scenarios I just outlined will help with the end-user demands as well. By externalizing BLOBs we are ensuring the load on SharePoint never gets too big, because if it does, then performance suffers, and the end-users complain about things like slow search times or the inability to find content altogether… and we all know who owns the complaint box!

Again, access is instrumental for the end-users and I think it is worthwhile mentioning that if you are going to archive SharePoint sites, folders and content, you have to make sure you don’t cut off end-user access to it. Ideally an end-user should be able to search for both content living natively within SharePoint sites and archived content that has been moved outside of SharePoint, and these actions must be executed through SharePoint search services. If the end-user has to open a separate tool (outside of SharePoint) to access content you have broken the experience, and with SharePoint it is all about the experience.

So, I trust you are in agreement that governing SharePoint is about governing the SharePoint information and the deployment, but more importantly, governing Information is about governing ALL information that lives in your infrastructure.

Your Information Infrastructure... More than just SharePoint

Elizabeth Kofsky — Thu, 7 Apr 2011 01:00:00 GMT-04:00

There’s no question that the amount of information in organizations today continues to grow, and grow exponentially. This probably doesn’t come to many as a surprise as all we have to do is look at our own organizations to see that content is everywhere. We all have email, file shares and now it would be fair to say almost all of us have SharePoint. With this said it is important to point out (especially considering that this will be but one of a series of blog posts around SharePoint) that for most organizations SharePoint is not the information infrastructure, it is in fact a part (to many varying degrees) of it.

Of course there are always exceptions to the rule. In smaller organizations, where perhaps content growth isn’t as big a concern or where SharePoint is the file share and Microsoft Exchange is being used with a few hundred users, SharePoint may very well fit the bill and be the entire information infrastructure. But in larger organizations with thousands of end-users, large email deployments, massive legacy file systems and network drives, existing records solutions, ERP/CRM tools, amongst other disparate repositories full of content… well, SharePoint is just part of the mix.

Suffice to say, for this series, we will address best-practices, challenges and issues organizations may face when SharePoint is part of the information infrastructure. More specifically I will try and focus squarely on how SharePoint can be a driver for good information governance practices within organizations.

When we think about Information Governance for SharePoint we need to think of two distinct sides of one single coin: governing the information and, maybe even more importantly, governing the solution. With that said, let’s start by discussing the reasons for governing the solution.

It would be an understatement to say that SharePoint is popular. Its adoption has been meteoric, and for good reason. SharePoint offers a simple to understand end-user experience, more so it connects seamlessly with another ubiquitous Microsoft solution, Office. And this isn’t to mention its strength in collaboration, bringing with it new heights of improved productivity.

But… Like anything gone unwatched, disaster is inevitable.

Yes, I am being a little melodramatic but for good reason. I’ve seen too many organizations deploy SharePoint disparately spread across their organizations only to end up with countless SharePoint sites living in literal anonymity from the data center, the eyes of IT. This can affect the corporate bottom-line very quickly, as storage hardware, software and administration costs can quickly expand. The good news: this is only a problem if you let it become a problem.

Ideally every organization needs to make sure it has a plan in place before they deploy SharePoint, but if that opportunity has come and gone there are still ways to assert a day-forward plan with clean-up support. To begin, it’s imperative a few rules are outlined around site creation and access. If you have control of when sites can be created and IT is at the center of site deployment and management you won’t have to worry about sites living disconnected from corporate control and information lifecycle management. Next of course is end-user access and content creation control. Either user enabled or automated, once rules are in place that help manage the growth of content, or automatically execute on an action (such as site archival) then we can better manage the volume of content and once again the information’s lifecycle. Most importantly, we can help to relieve one of the many IT headaches related to the holistic management of the information infrastructure.

In summary, we need to make sure we are leveraging the very powerful and productive benefits that SharePoint brings to the table, but in doing so we also have to keep in mind that although we will actively use SharePoint more than other content-centric solutions it must be well integrated as part of our bigger information infrastructure. In order to do this, a great starting point is to define a strategy that ensures you are governing the deployment of SharePoint Sites and users.

Our next blog post will take a deeper dive into the other side of the coin mentioned, governing the information itself!

Government Appeals Judge Scheindlin's FOIA Order on Metadata

Stephen Ludlow — Sun, 13 Mar 2011 11:42:00 GMT-04:00

Apparently I wasn't the only one to see the potential implications of having to produce metadata in FOIA requests.

It appears that the US government has done its homework as well. In a Feb. 21 motion, the government appealed and is seeking a stay of Judge Scheindlin's FOIA Order on Metadata in NDLON v. ICE

There are a number of juicy tid-bits in the government's lengthy (26 page) argument that demonstrate that they too have determined that this ruling could have a substantial impact on the FOIA process:

Given the sea change in FOIA practice that could result from this Order, the Second Circuit should have an opportunity to conduct a full review before the Government is required to produce the records that are the subject of the Order. (page 3)

From my perspective, this one pretty much says it all. The US Government sees that having to produce metadata for FOIA requests will have a monumental effect on the way that they responds to FOIA requests. And just in case this could be more narrowly construed as a "sea change" in FOIA practices for just this case, it is further emphasised at the beginning of arguments…

In light of the enormous potential implications of the Court's Order for FOIA practice throughout the country, as well as the Government's determination to appeal that Order, a stay should be granted pending appeal. (page 16)

And then again in wrapping up arguments

...the Court issued a decision with potentially sweeping consequences for FOIA practice throughout the country... (page 25)

Not only does the government obviously understand that the requirement to produce metadata will significanty change their business process, they also see the potential for the same high costs that are associated to eDiscovery.

ICE obtained permission to use an e-discovery program called Clearwell for FOIA purposes, at a cost of over $270,000 in upgrades, and the acquisition of a $32,000 server. See Pavlik-Keenan Decl., 7-8, 11. Continued use of the software will require an additional expenditure of funds in an unknown amount, perhaps in the hundreds of thousands, if not millions, of dollars. (pgs 23-24)

The eDiscovery application that has been used for culling, processing, and the creation of loadfiles is Clearwell in this case, but it is evident that the Immigration and Customs Enforcement Agency was not prepared for eDiscovery and is not prepared for the potential costs - or the potential risks.

See id. 13. In addition, ICE was required to suspend many of the agency's security protocols in order to allow Clearwell to run properly. See id. 11. This is further evidence of irreparable harm that entitles the Government to a stay of the Order pending appeal.

As they say, this is going to get interesting. eDiscovery is already creating a "sea change" when it comes to the way that corporate America is dealing with its vast store of unstructured electronically stored information. Could Judge Scheindlin's opinion be the harbinger of this same kind of change in government? Let's stay tuned to see the results of the appeal!

Die Hard

Jeremy Barnes — Tue, 15 Mar 2011 03:07:00 GMT-04:00

In his latest Executive Brief, “Getting Started with Information Governance” Barclay Blair identifies five fundamental, practical activities companies should undertake along the way. The last one in particular caught my attention.

The really hard problem. Managing information requires cultural change. Cultural change is tough, and will not happen unless it is driven from the top. If senior management does not lead this change in a real and personal way, enterprise IG will not happen. Plan accordingly.

In the past two weeks I’ve been out and about visiting a couple organizations and talking about information governance and email management. One of the companies is a long-time customer of ours, three years into an email management project. The other was a prospective customer, just beginning to think over the thorny topics associated with their email management program. Despite the vast difference in terms of what stage each company was in, a common subject underscored much of the conversation: user behavior and cultural change. Questions always arise like, what can we do within a solution to facilitate a necessary cultural shift, and more importantly, how can a solution encourage the proper behavior to ease such a transition?

Considering these discussions concurrently emphasized for me that this “really hard problem” – as Barclay says – obviously isn’t about an immediate change. But even three years in (and with marked success and ongoing commitment) companies find that old habits die hard. Plan accordingly.

In a real and personal way

Barclay remarks, “if senior management does not lead this change in a real and personal way, enterprise IG will not happen”. Executives are busy folks – but some days I have the good fortune to see one seated in the room when I walk in brandishing a visitor badge. Common sense suggests the ensuing conversation is very business level – focus on the ROI and the soft benefits like risk reduction.

Yet inevitably, in discussions of email management, I find that even the highest ranking individuals in a company are eager to focus in on the most granular of features. “What does that button do? Why doesn’t it work this way? But I would rather do this…no one would do that!”

It’s counter-intuitive to expect a senior executive to get bogged down worrying about features and functions – but it emphasizes how “personal” information, email in particular, really is. Even up to the C-level, if something different is going to happen with email, everyone will have an opinion (and it better not affect my BlackBerry!). When getting started with an information governance project, don’t underestimate the likelihood of people having opinions – even about the most granular details. Use this opportunity as a means to underscore the importance of cultural change.

To the point

It’s for this reason that we afforded such attention to user experience in the OpenText Email Management for Microsoft Exchange product. A couple fundamental tenets underscored our research & development exercise:

Integrate cleanly – Integrate so closely with the productivity tools in use today (e.g. Microsoft Outlook) that users are only peripherally aware – if at all – that a formal content management system is behind the scenes and actually controlling and managing their business relevant content. Maybe even allow users to access managed emails from the SharePoint interface, further allowing individuals to work within their productivity tools of preference and simplifying the user experience.
Understand behavior – Provide multiple mechanisms by which individuals can participate in the processes of information management. Considering the (what now seems age-old) principle of “pilers and filers” in the world of email users, understand that people simply work differently, and email management solutions should accommodate the way that people like to work. Email management solutions should not require them to adopt a completely different methodology just to accommodate the information governance directive.
Be flexible – Anymore it’s increasingly less likely that individuals are sitting at a desk cleanly connected to the corporate network. Tools like BlackBerrys and iPhones, and interfaces like Outlook Web Access characterize how many individuals interface with their business information. Email management solutions must accommodate such paradigms, and evolve in turn.

Paired with the support of senior management, an accommodating product is fundamental to achieving user adoption and ultimate acceptance.

Side-stepping knee-jerk investments

It’s not easy to change. But the environment in which our businesses are expected to operate will continue to demand it. Look back over the SOX aftermath, consequent FRCP amendments, and more recently, Dodd Frank – all external factors driving change in our information management practices. How we react says a lot about our corporate culture. It’s (relatively) easy to make knee-jerk technology investments like those that companies made in traditional email archives in years past - archiving all email and keeping it indefinitely. Investments like these are often the ones users most readily adopt, but have no real impact on our ability to manage the risks and costs of information. It is our responsibility as practitioners and advocates of information governance to find the balance between instituting change successfully, and achieving the positive impact ushered in by better information management.

Like Barclay states, the oversight and support of senior management is imperative. But don’t let the direct involvement of senior management simply be a catalyst for inflating the functional requirements in the RFP. Instead allow it to instigate – and subsequently justify – a broader discussion around the effect and impact of information governance on corporate culture.

Visit Barclay's "Essays in Information Governance" blog and have a look at the full brief.

The Impact of Producing Metadata in FOIA Requests

Stephen Ludlow — Tue, 8 Mar 2011 12:56:00 GMT-04:00

I know, I know, I promised the second part to Information Governance & the Law of Unintended Consequences, but I did want to share some commentary on the recent Judge Scheindlin ( of Zubulake v. UBS Warburg and Pension Committee fame) ruling on the production of metadata in a US Government Freedom of Information Act (FOIA) request before it fades out of the news. Information Governance & the Law of Unintended Consequences part II is coming soon.

The gist of the Nat. Day Laborer Org. Network v. United States Immigration and Customs Enforcement Agency ("NDLON"), 2011 WL 381625 (S.D.N.Y. Feb. 7, 2011) ruling is that for large productions, the government should be held to the same minimum standards routinely set out by government entities such as the SEC when requesting information from corporations. This standard includes "TIFF image format but with corresponding load files, Bates stamping, and the preservation of "parent-child" relationships,' along with metadata typically produced for documents and email.

Dean Gonsowski at Clearwell, has done a great job of summarizing the legal nuances and implications, so I thought it might be interesting to think about the potential impact of this ruling on government agencies and departments and the processes that are used to respond to FOIA requests.

First, let me preface this by saying that I don't have first-hand experience on how US Federal Agencies respond to FOIA requests. However, being Canadian and working in Ottawa for the last 20 years, I have experience on the Canadian equivalent to FOIA, Access to Information and Privacy (ATIP). The way both countries deal with requests is similar enough that there are Case Management solutions that service both markets. My commentary is based on this experience,

Responding to a FOIA or ATIP request is largely an exercise in Case and Correspondence Management, and much less like the process (www.edrm.net) of responding to an eDiscovery request that many of us are now familiar. This is largely due to the nature and heritage of these requests. In the past, departments dealt with a high volume of requests, but a low volume of responsive documents. This was due to the heavy reliance on paper for government record keeping.

As information has become more and more digital, the FOIA/ATIP process has been somewhat adapted to include electronic documents, but basically maintained its heritage as a paper-based process.

Today, the typical collection and production process involves:

Contacting the custodians of information to request relevant document
The custodian searches for, and prints relevant documents
Printed documents are typically handed over to a centralized group responsible for managing the redaction of sensitive information and the approval of released document
Printed document are either scanned as images for electronic routing, or physically routed in file folders
Paper documents are redacted by hand using a highlighter
Electronic document are redacted using a redaction program
Documents are routed for approval on redaction and inclusion
Approved paper documents are copied using a special photocopier that blacks out highlighted text
Redacted electronic image documents are printed out
Content is routed to the requester

As you can see, this a very paper-centric process. It is also a process that makes it nearly impossible to maintain the metadata that Judge Scheindlin proscribes should be included in any large scale productions. The impact that requiring metadata in large FOIA requests could have on government could be substantial.

Requiring metadata brings the largely separate and independent FOIA process and places it directly on a converging path with eDiscovery - and many of the issues, costs and risks associated to eDiscovery as well.

Self-Collection

I'm sure there was a collective shudder from eDiscovery Gurus when I mentioned the first step in the collection process was for end-users to self-identify and print content that might be relevant to the FOIA request. I won't bother to rehash the issues associated to self-collection, but given the importance of maintaining metadata, obviously the existing process of printing documents won't work. Additionally, due to the growth in the importance and volume of content that is only in electronic format, expecting end users to be able to search for, and find all relevant information is an increasingly difficult position to defend. Government agencies and departments will need to turn to the same tools that are being adopted in many organizations in the corporate world. Early Case Assessment applications to automate the collection process based on keywords, metadata filtering and date filtering from a wide variety of datasources.

Processing

By introducing the need to be able to produce metadata, Judge Scheindlin will force the issue of processing. Processing is the activity associated to opening documents and archives to extract attachments, embedded objects, the metadata, and the full text so that documents can be loaded into a review platform. Additionally, documents are often converted to image files (TIFF or PDF) either before being added to the review platform, or as part of the review process prior to final production. For the groups responsible for FOIA and ATIP requests, the processing of content is a completely new activity. Departments and agencies will need to determine if they will develop the processes and deploy the tools internally to Process content, or if they will do what many organizations do, which is to incur the $300 - $1000 per GB price-tag typically associated to outsourced Processing.

Review and Approval

There is also an opportunity to apply much of what has been learned in eDiscovery over the past few years to help shift the way things are done in approving document for release for FOIA and ATIP requests. Once again, it is the large volume of Electronically Stored Information (ESI) that will force organizations to reevaluate their processes. With the sheer volume of thousands, if not millions of documents, there can be no expectation of single approvers approving every document. Similar to eDiscovery Review, the initial redaction and approval ESI can be performed by specialized resources, with the ability to flag documents that will require additional, higher levels of approval. Additionally, the same rigour that is applied to the final Quality Assurance review can help to summarize and test the large body of information that is to be disclosed.

Information Governance

It always comes back to Information Governance! Whenever we begin to look at business processes that are impacted by the volume and organization of Electronically Stored Information, whether it be eDiscovery, FOIA, audits, regulatory requests, or even storage, there will always exist the potential for improvement through the use of good Information Governance. While I could certainly provide more complex definitions, I think the easiest way to consider governance is to look at it as striking a balance between the Risk, Cost and Value in the creation and storage of information. I think that both the US and Canadian Federal Governments have made a good start when it comes to managing Electronic Records. However, the increased burden of metadata productions (not to mention eDiscovery) has placed a price on ALL the information in the organization. Like their corporate counterparts, government will need to evaluate their information governance policies to determine where risks and costs can be reduced. Finally, good information governance, that includes enforceable policies, security, and audit can also lead to greater confidence and transparency. Greater transparency can actually lead to fewer FOIA requests.

So I think at this point we should take heed of a Judge Scheindlin footnote:

"To be clear, my Order requiring the use of this Proposed Protocol for future productions-as amended by the specific metadata fields I have required and by the options I have offered the parties regarding the form of production for spreadsheets-is limited to this case. I am certainly not suggesting that the Proposed Protocol should be used as a standard production protocol in all cases. The production of individual static images on a small scale, where no automated review platform is likely to be used, may be perfectly reasonable depending on the scope and nature of the litigation."

In other words, she strongly warns us "horses for courses" when it comes to FOIA productions - and like in eDiscovery, she stresses the importance of cooperation and early definition of expectations in meet and confer sessions. I think this same spirit of reasonableness will likely permeate lower levels of US government, and like eDiscovery, it will likely begin to radiate to other countries.

If Judge Scheindlin's ruling continues to pick up steam and more FOIA requests include large volumes of ESI and the requirement for metadata, government organizations will have to adapt new polices, processes and technology. This will require a significant investment on the part of government. While it is early days for planning this kind of investment, my prediction is that by this time next year, many government agencies and departments will be considering more closely aligning their FOIA response programs with their eDiscovery programs.

Information Governance & the Law of Unintended Consequences

Stephen Ludlow — Fri, 25 Feb 2011 07:31:00 GMT-04:00

To quote Wikipedia, "The law of unintended consequences is an adage or idiomatic warning that an intervention in a complex system always creates unanticipated and often undesirable outcomes." Certainly, if there was ever a complex system, it is the relationship that large organizations have with the vast amount of electronically stored information (ESI) that is created by today's knowledge workers.

The complex system that organizations operate within includes an environment that is now more heavily regulated, where litigation is on the rise and where the pressure to cut costs has not diminished during this period of anaemic economic recovery. Inside the organization, knowledge workers are creating and storing content at an unprecedented rate, not just in traditional enterprise applications like email and office productivity suites, but also in new authoring paradigms such as social media applications and video. This content is creating IT issues, legal issues, compliance issues and even issues for the business and knowledge workers themselves.

It is in the response organizations are taking to manage the issues associated to content that we are seeing the law of unintended consequences played out time and again.

The Unintended Consequences of Keeping Everything

The early adoption of email archiving was probably the poster child for unintended consequences. Around five years ago, email archiving was the last, great IT project where IT could implement technology without any significant consultation with the business side of the organizations. IT looked at their email environment and realized that end user were using email as a content and knowledge store (dump), and that the large volume of email was negatively impacting email server performance, back-up times, and storage costs. IT often became instant heroes by introducing Email Archiving. Emails with large attachments and emails over a certain age could be archived and IT could eliminate mailbox quotas, demonstrate significant storage savings and maintain or improve email performance with existing infrastructure, all with very little negative impact on end users.

It actually took a few years and a change in the U.S. Federal Rules of Civil Procedure for the unintended consequence to become evident. In most organizations, without any guidance from Legal and policies from Records Managers, IT made the decision that it was easiest to simply keep email forever. Archive storage seemed cheap and end users seemed happy. Unfortunately, two things happened. The first was the compound growth of content. 10 TBs quickly became 40 TBs which became 70 TBs with no end in sight. The second was the impact of legal discovery. Legal quickly learned that the archive was a rich source for email and placed content in the archive on litigation hold. The result was a morass of content, still growing at an exponential rate, full of risky content that might have been deleted as per retention policy had those policies been applied. Because it was impossible to assess what might be on hold, what might be a record, and what might be defensibly deleted, many organizations continue contribute to the problem. They also continue to contribute to the cost and the risk - certainly not the initial intention of the archiving program.

The Unintended Consequences of Getting Rid of Everything

End-users are those pesky things that stand in the way of the best laid plans when it comes to managing content. That is certainly the case when it comes to the approach many organizations have tried to reduce the risks and costs of content. Due to the risk of needlessly retaining the so called "smoking gun", along with the cost associated to collecting, processing and reviewing large volumes of content, many legal departments advocated rather severe retention policies. The policies require users to declare official records of the company, and get rid of everything else in a short period of time - typically between ninety and one hundred and twenty days. The intent was to reduce the volume and get rid of much of the conversational content that email so typically includes. Unfortunately, in practice, end-users place value on this type of content - whether it be as a record of decisions made or as reference material. The result of a policy of getting ready of everything has typically been end-users circumventing policy by storing email in local archives, the desktop, thumb-drives and even sending email to personal accounts. The unintended consequence is actually information more randomly sprinkled throughout the organization. Possibly even worse, the organization becomes riddled with end-users knowingly circumventing company policy, a position that is difficult to defend in the courts.

Information Governance

Information governance helps organizations to manage and balance the regulatory requirements, risks and costs of the content with the need to ensure that knowledge workers are productive in the face of exponential growth of content. The number and impact of unintended consequences can be reduced through information governance by bringing balance to the differing needs of IT, Legal, Records Managers and the Business. Organizations can start by bringing stakeholders from each of these areas together to define requirements and understand the requirements of other stakeholders. Using this as a starting point, organizations can develop balanced policies, practices and can determine what technology best supports these policies and practices.

No two organizations are alike. Each organization has its own culture, internal policies, and exists in different regulatory environments. This means that every organisation will approach information governance differently. Conservative organizations, typically found in government and highly regulated industries, lean towards keeping more Electronically Stored Information. Organizations less regulated, but more frequently litigated, often lean towards getting rid of more information, requiring users to take action on the content they want to retain beyond the basic retention period. Sometimes, organizations even display this dichotomy at the departmental level. When evaluating technology for information governance, it is critical that organizations understand the impact the technology will have on their ability to meet their objectives, and that it is flexible enough to not introduce its own unintended consequences.

Coming soon, Part II - Unintended Consequences and the importance of information classification.

Additionally, a quick shout out to Barclay Blair, who touched on a number of these topics in a much more eloquent fashion - but then, my wife is not a contemporary artist. Barclay is also running a series of Executive Briefs with OpenText. Worth checking out.