The eDiscovery Paradigm Shift

Integreon Announces Single Source eDiscovery

noreply@blogger.com (Charles Skamser) — Tue, 22 Jul 2008 10:38:00 -0500

In a followup to my recent post from July 2, 2oo8 titled "eDiscovery Single Source Solutions", Integreon Managed Solutions, Inc., the global leader in integrated Knowledge Process Outsourcing (KPO), has confirmed the interest in providing single source solutions by announcing the launch of “Doctane”, a complete discovery management solution, which simplifies litigation project management, reduces the total cost of litigation and improves the predictability of litigation expense budgets.

Doctane offers a full suite of discovery services which can be integrated or selected as stand-alone components, at a fixed price per document including: Discovery Management Consulting; Data Collection, Processing, Analysis and Early Case Assessment; Data Hosting; Document Review and Production. Doctane is operated entirely in-house by Integreon, so clients benefit from the “one stop” convenience of managing all their discovery requirements with a single service provider.

Integreon has chosen MetaLINCS, Attenex, Concordance FYI and CaseLogistix as its best-in-class technology partners and Offers the flexibility of rapid start, cost control, and round-the-clock review teams with a choice of in-house, full-time staff attorneys to conduct privilege and responsiveness reviews from wholly-owned domestic centers in New York City, Fargo, ND (USA) and offshore centers in Mumbai and New Delhi (India), and Manila (Philippines).

As previously stated, I believe that vendors such as Integreon that go down the single source solution route are absolutely on target with market demand and as such will be one of the survivors as the market shakes out the pure players. However, from a users perspective, I would be more concerned with the risk associated with a player like Integreon that is taking a best-of-breed technology approach and is therefore at the mercy of the actually software vendor than with a player such as Case Central, Autonomy or Clearwell Systems that actaully owns their own technology.

Please note that as my next followup to eDiscovery Single Source Solutions, I am planning to review The OneO® Discovery Platform, an recently announced offering by Orange Legal Technologies out of Salt Lake City.

eDiscovery Single Source Solutions

noreply@blogger.com (Charles Skamser) — Wed, 02 Jul 2008 09:37:00 -0500

Over the past twelve (12) months months I have spent what seems like eighteen (18) months meetings with litigators from law firms, General Counsels from the Fortune 5000, owners and operators of litigation services providers, visionaries from the litigation technology vendors and investors that are either in the litigation technology services space or interested in taking a position in the space.

And although the opinions vary widely regarding the most important issues that the industry faces and the direction that the industry will take, the one common theme that has bubbled to the top in every discussion is the debate over "single source" solutions as opposed to "best of breed" solutions.

Having spent twenty five (25) years in the enterprise class software market selling multi-million dollar solutions to the Fortune 5000, this is a very familiar debate and therefore I recognize and understand the issues on both sides of the argument.

The Users Perspective and General Observations
From the users perspective in any market, they want solutions that addresses their requirements for a fair price. At this very high level of requirement, the ultimate solution could be either a "best of breed" solution or a "single source" solution. However, when users start to investigate the details, the differences begin to become very real.

Working with "best of breed" solutions requires users to spend lots of resources building customer solutions that ties them into old technology. In addition, as time goes on and market requirements change, these solutions require expensive maintenance and in some cases don't enable users to keep pace with demand.

Working with any of the newer solutions requires patience and reliance upon either poorly integrated older technologies or unproven leading edge technolgy. So, in both cases their are pros and cons, sacrefices and trade offs that will have to be made.

In the end, users in the eDiscovery market will pay for repeatable and stable results. In some cases this will come from "best of breed" solutions and in some cases this will come from "single source" solutions. It just depends upon what the user is willing to sacrifice for the required outcome.

The Vendors Perspective and General Observations
Vendors in this debate usually fall into one of two categories. Those vendors that can provide single source solution and those that cannot.

The vendors that can provide or are attempting to provide a "single source" solution will be tauting the financial and operating benefits of only having to deal with one vendor from both a procurement and service standpoint. Historically, the first vendors to offer the "single source" solution in any market, started with a strong technical foundation in that market with a leading product and then acquire and integrate the peripheral technologies or solutions required to complete a "single source" solution.

The benefits of this initial entrants into the "single source" fray is that they do have a solution (no matter how clumsy), they will normally have the financial resources to make it work (i.e. lots of engineers and field service personnel) and they usually have referenceable clients.

The shortcomings with these initial "single source" solutions are that they are usually built on older technology (i.e. client/server or worse), the integrations are not particularly well done, the are usually not easy to use and they are expensive to maintain (i.e. and therefore expensive for the users). Common traits will be the requirement to store data in multiple disparate databases, the requirement for lots of manual operations and user interfaces that are trying to appear to be something that they are not (i.e. Citrix based emulations vs. true SaaS based HTML GUIs).

Examples of this group in eDiscovery would be vendors such as FTI and Autonomy that are currently trying to assemble a "single source" solution through MA activities.

Shortly after the emergence of this first vendor group, true "single source" solutions will begin to appear that were designed and built specifically to provide a truly seamless single source solutions. These vendors are normally start ups relying on very leading edge technology (e.g. SaaS), possibly venture backed (i.e. not as many in today's economy), have immature sales organizations and very few if any real referenceable clients. They are the prototypical study in Geoffrey Moore's Crossing the Chasm.

The benefits of this second group is that they usually have very elegant solutions, they are normally much less expensive then the first vendor group, they are willing to provide lots of perks for using their technology and users will usually be dealing with the founders and developers of the technology.

The shortcomings of this second group is that they are offering new and unproven technology, they may not have the capital to sustain themselves (i.e. unless they have venture funding), they may in fact not have the best solution in every category and the technology may go through some growing pains as it matures under the rigors of the real world.

An example of this type of vendor in the eDiscovery market would be CaseCentral.

The vendors that cannot provide a single source solution are easy to spot as they will use terms such as "pure play" and "best of breed" and discuss the ramifications of not having the very best solution in any of the required sub components. These vendors are normally very large and established and are usually very good at one specific component of the entire requirement. This vendor group will normally also have a very large, active and expensive custom integration consulting team that will be able to integrate their technology with just about anything that you are willing to pay for.

The benefits of working with these vendors is that users will in fact get the best, most widely tested solution for whatever component that vendor provides. And, for the larger Fortune 100 clients, they may in fact be able to assist in assembling a true "best of breed" custom system.

Other Opinions
As evidence that this issue is becoming "ripe", we are starting to see a variety of opinions appear. A good example is a recent post by Barry Murphy on his "Blogging at the intersection of archiving and eDiscovery" Blog titled, "Best-of-Breed or All-in-One Solution for eDiscovery? ". Barry isn't completely sold on the concept of "single source" solutions. However, he seems to be warming up to the idea.

Summary
In summary, I have purposely not come to any conclusion regarding this debate in eDiscovery because I want to continue to study the market. However, in case it wasn't obvious, based upon my experiences in the enterprise class software markets, I am a big proponent of "single source" solutions. I understand the inherent limitations and risks for users but also understand the tremendous rewards.

So, send me you comments and opinions and stay tuned for my next update on this topic.

eDiscovery Search Predictions for 2008

noreply@blogger.com (Charles Skamser) — Sat, 14 Jun 2008 10:36:00 -0500

As I continued my education eDiscovery search platforms over the past couple of weeks and subsequent update to my eDiscovery Paradigm Shift Blog, I came across a really interesting update by Stephen E. Arnold on the current state of the search market in general titled "Search Rumor Round Up, Summer 2008". And, although it is not specific to search in the eDiscovery space, his overview of search is outstanding and very applicable to what we have to look forward to in eDiscovery.

As I pointed out in my post titled "eDiscovery Search Case Law Emerging", the courts are starting to catch up in regards to the value and impact of search in the eDiscovery process along with the subtle nuances of search technology. And, as I talk to law firms and the legal departments of many of the Fortune 500 about their eDiscovery issues, I am finding that the topic of search and more recently conceptual search is being raised more often in the context of culling, de-duping, finding potentially responsive and privileged docs and gaining a better understanding of their data in general. However, I have also found a complete lack of understanding of current search technology and it applicability to the real needs of the litigators and their litigation services consultants.

Further, I am curious to understand the foundation for this weeks acquisition of Attenex by FTI. Was this a fire sale because the conceptual search market has not matured and therefore Attenex has not been able to reach its full potential as a conceptual search based review platform? Or, was this a brilliant move by FTI to add yet another leading edge technology to its roster based upon accelerating market demand? Just for the record, I happen to think that it is the latter. But, there are some intriguing arguments for the former.

The observations and comments in Mr. Arnold's article that I believe are most applicable to eDiscovery are as follows:

Rumor 1: More Consolidation in Search
As eDiscovery technology matures and the obvoius winners in technology and the appropriate strategy/formula for success begins to emerge, we are seeing the same basic consolidation in eDiscovery in general and will continue to see even more eDiscovery technology consolidation over the remainder of 2008 and in to 2009.

Rumor 5: Search Will Become a Commodity
I believe this prediction to be true at the desk top / SaaS based eDiscovery platform level where the user is wanting to search several 100,000 docs in a short period of time. However, at the service center production level where the document pool is terabytes of data, I believe that there is still room for several search technology leaders that can figure out how to get these massive search project done in hours as apposed to days.

Rumor 6: Search Is a Component of Other Enterprise Software
I believe that there is no doubt that user are quickly going to expect sophisticated search to become a seamlessly integrated part of any eDiscovery platform.

Rumor 9: Key Word Search Is Dead
This is my favorite prediction for eDiscovery as I can visualize all the current eDiscovery vendors cringing.

Rumor 10: A Hardware Maker Will Put Search on a Chip
Coming from a background of integrating the appropriate and ripe software technologies into firmware / hardware solutions, I absolutely agree with this prediction. And, it fits really nicely with the "behind the firewall appliance" direction that some of the archiving vendors are going.

All of this being said, the full text of Mr. Arnold's article is as follows:

I am fortunate to receive a flow of information, often completely wacky and erroneous, in my redoubt in rural Kentucky. The last six months have been a particularly rich period. Compared to 2007, 2008 has been quite exciting.

I’m not going to assure you that these rumors have any significant foundation. What I propose to do is highlight several of the more interesting ones and offer a broader observation about each. My goal is to provide some context for the ripples that are shaking the fabric of search, content processing, and information retrieval.

The analogy to keep in mind is that we are standing on top of a jello dessert like this one.

The substance itself has a certain firmness. Try to pick it it up or chop off a hunk, and you have a slippery job on your hands. Now, the rumors:

Rumor 1: More Consolidation in Search
I think this is easy to say, but it is tough to pull off in the present economic environment. Some companies have either investors who have pumped millions into a search and content processing company. These kind souls want their money back. If the search vendor is publicly traded, the set up of the company or its valuation may be a sticky wicket. There have been some stunning buy outs so far in 2008. The most remarkable was Microsoft’s purchase of Fast Search & Transfer. SAS snapped up the little-known Teragram. But the wave of buy outs across the more than 300 companies in the search and content processing sector has not materialized.

Rumor 2: Oracle Will Make a Play in Enterprise Search
I receive a phone call or two a month asking me about Oracle SES10g. (When you access the Oracle Web site, be patient. The system was sluggish for me on June 14, 2008.)The drift of these calls boils down to one key point, “What’s Oracle’s share of the enterprise search market?” The answer is that its share can be whatever Oracle’s accountants want it to be. You see Oracle SES10g is linked to the Oracle relational database and other bits and pieces of the Oracle framework. Oracle’s acquisitions in search and retrieval from Artificial Linguistics more than a decade ago to Triple Hop in more recent times has given Oracle capability. As a superplatform, Oracle is a player in search. So far this year, Oracle has been moving forward slowly. An experiment with Bitext here and a deployment with Siderean Software there. Financial mavens want Oracle to start acquiring search and content processing companies. There are rumors, but so far no action, and I don’t expect significant changes in the short term.

Rumor 3: Microsoft Will Tidy Up Its Search Operations
This rumor suggests that Microsoft, a giant company with many barons and dukes controlling fiefdoms, can deploy one search solution. I don’t think that will happen quickly. The Certified Gold Partners who make better search systems than those available from Microsoft can rest easy for the foreseeable future. Search is too complicated in general and within Microsoft for a one-size-fits-all solution. I anticipate more search options, not fewer. Coveo, Exalead, ISYS Search Software, and others will benefit from the Microsoft approach to search for months, if not years.

Rumor 4: Semantic Search Will Unseat Google
Semantic technology is now within reach of almost any search and content processing vendor. The technology is relatively well known and the processing power is available at a reasonable cost. By itself, semantic search will not be enough to shift the market share that Google is amassing in the consumer search and enterprise markets. Google’s been chugging along for a decade, and it has yet to meet significant competition other than itself. Semantic technology is a component, not a Google killer in the hands of a competitor at this time.

Rumor 5: Search Will Become a Commodity
No, as I described in my Web log post on May 12, 2008, about the “search elephant”, search has too many different meanings for one solution to sweep the board. Each unit of a company has many different search and content processing needs. It is, therefore, difficult to convince the legal department to use the open source Lucene tool for eDiscovery. The legal eagles will want to use a service from Brainware or Stratify. Down the hall, the chemical engineers need to find chemical structure. Search consists of niches, and these will bump heads, overlap, and been quite confusing to sort out. In that confusion, consultants and different vendors thrive.

Rumor 6: Search Is a Component of Other Enterprise Software
This is a rumor related to “search will become a commodity”. True, enterprise software vendors will include more robust search and content processing systems in their software, market the heck out of the enhancements, and bundle it with whatever the client wants to buy. But enterprise applications open the door to point solutions that meet specific needs. So search certainly will become ubiquitous and the ecosystem will spawn new species of information access. Nope, search is going to be with us for a long, long time.

Rumor 7: The Google Search Appliance Doesn’t Work
False. The GOOG has more than 10,000 licensees, a fleet of partners, and the OneBox API that can make the Google Search Appliance work like Roy Roger’s prescient horse, Trigger. The GOOG has had an impact on the enterprise search market. It’s easy to complain about the Google Search Appliance. It’s harder to explain how a company with such an interesting approach to sales can sell such a large number of units. Obviously a certain sector of the market wants these Google boxes.

Rumor 8: Social Search Will Revolutionize Enterprise Search
Nope. Social functions can be useful, but in regulated industries, there are some challenges associated with social search. Social search is the equivalent of a restaurant’s weekly special. If the customers gobble enough of the dish, the special could be promoted to a specialty. It’s early days for social search in an enterprise, but it’s not too soon for law enforcement and military intelligence people to embrace the concept. Social search is quite useful in certain areas, but one needs to have lots of social data to crunch to see the technology in full flower.

Rumor 9: Key Word Search Is Dead
Key word search is hard for many people. Alternatives and options are needed. But key word search is too useful in certain types of research to go the way of the dodo. Investors like to think that a whizzy interface without a search box is the next big thing in search. Interfaces are becoming more important by the hour. But an interface without a way to look for words and phrases won’t carry the day.

Rumor 10: A Hardware Maker Will Put Search on a Chip
What’s happening is research and investigation. The Exegy appliance could be boiled down to a smaller gizmo. At some point in the future, any search appliance could be reduced to firmware. I think the likelihood of search on a chip is high, but it’s not something you will be able to buy in 2008. Intel invested in Endeca for a reason. Intel had a brief love affair and then a messy divorce with search vendor Convera years ago. Other chip-centric outfits are poking around in this area as well. On the horizon, yes, but appliances will be about as close to search in a single package that we will have in 2008.

Observations
Most people don’t realize that search is like a giant jello dessert. There is a shimmery, attractive quality to the whole thing. When you start to pick it apart, the substance becomes slippery and tough to pin down. It’s easy to be fooled by surface changes like semantic search and social search, which are like a squirt of whipped topping on the jello. Do you have candidates for rumors you think I should have included in my round up. If so, use the comments section of this Web log to post your favorites. To avoid legal hassles, I may have to edit some of your inputs. Ah, life in the modern world is so rewarding.

Concept Search Case Law Emerging

noreply@blogger.com (Charles Skamser) — Thu, 12 Jun 2008 19:43:00 -0500

In a followup to my post regarding my investigaiton of a SaaS based conceptual search technology for the eDiscovery market: http://ediscoveryconsulting.blogspot.com/2008/05/in-search-of-integrated-conceptual.html, I came accross an interesting article on the Law.com site by Leonard DeutchmanPennsylvania Law Weekly titled "When E-Discovery Is Put to the Test, Will federal rules on expert testimony govern admission of search engine results?".

This outstanding article discusses the issues of how the courts view search technology in light of Disability Rights Council v. Washington Metropolitan Transit Authority, 242 F.R.D. 139 (D.D.C. 2007) and United States v. O'Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008) and Equity Analytics v. Lundin, 2008 U.S. Dist. LEXIS 17407 (D.D.C. Mar. 7, 2008).

Please note that I am still evaluating search technology and will post my finding as soon as I have completed my investigations. However, I felt as though this article raised so many pertinent and timely issues that I wanted to post it before my findings were complete.

The fulll article is as follows:

An influential federal district judge whose opinions on e-discovery are well respected may have set e-discovery on a path toward its most searching scrutiny yet.

In Disability Rights Council v. Washington Metropolitan Transit Authority, 242 F.R.D. 139 (D.D.C. 2007), Judge John M. Facciola recommended "concept searching," -- the use of complex search engines that make use of linguistic or statistical patterning to locate responsive e-mails and electronic -documents, in order for a tardy producer of discovery to wade through voluminous electronically stored information quickly. Interestingly, Facciola made no mention of whether the use of concept searching tools should be subject to Federal Rule of Evidence 702, which governs the admission of scientific or expert testimony.

Recently, however, in United States v. O'Keefe, 537 F. Supp. 2d 14 (D.D.C. 2008) and Equity Analytics v. Lundin, 2008 U.S. Dist. LEXIS 17407 (D.D.C. Mar. 7, 2008), Facciola held that any challenges to or defenses of search methodology in producing e-discovery must be scrutinized under Rule 702, and so ordered hearings under Daubert v. Merrill Dow Pharmaceuticals, 509 U.S. 579 (1993).

These rulings give rise to the question of what a Daubert hearing for an e-discovery search engine would look like.

HOW AND WHERE TO SEARCH
The first issue the court would address is how search engines search. The most direct approach is keyword searching, which take three basic forms:
Direct searching for keywords, e.g., "Locate all files with 'Jones.'"
Boolean searching, e.g., "Locate 'Jones' or 'Smith'," "Locate 'Jones' but not 'Smith,'" and other combinations.

Proximity searching, e.g., "Locate 'Jones' within 25 words of 'Smith.'" Often such searching is restricted by date range, e.g., "Locate all e-mails with 'Jones' created after January 1 but before July 1, 2007 only."

Concept searching, as has already been briefly discussed, takes a different approach. It targets information relating to a concept even if specific keywords are not present (e.g., a series of e-mails mentioning the words "Clinton," "McCain" and "Obama" would likely concern the 2008 U.S. presidential election, even if the phrase "presidential election" does not appear).

Some concept searching tools use "taxonomies" or "ontologies," that is, compilations of both commercially available data and data supplied by the client pertinent to the case collected from the lawyers and key players. Some concept searching uses linguistic analysis examining how the communicants discuss matters, while other approaches, such as "clustering" and "latent semantic indexing," use mathematical probabilities to determine whether a given file is related to a given concept. For an excellent discussion of concept searching, see "The Sedona Conference Best Practices Commentary on the Use of Search and Informational Retrieval Methods in E-Discovery."

A DIFFICULT HEARING
Regardless of which approach the search engine takes, the actual Daubert hearing will prove difficult for two practical reasons, both stemming from the fact that search engine applications are proprietary. First, it simply will be hard to get the designer to appear at the hearing to testify as to how the engine works. Second, the designer will fight giving the best evidence of the efficacy of the engine, that is, the engine's source code, because that code is proprietary. Should the code be revealed, the design would lose its value, as anyone could use that code without having to obtain a license (i.e., a copy of the application) from the designer.

Proprietary applications can be validated without their source code revealed, but only under certain circumstances. The easiest is where a specific positive finding needs to be corroborated. For example, if a proprietary forensic search tool such as Guidance Software's EnCase reports that a file is found at a particular location on a hard drive, an examiner can use an "open source" tool, i.e., a tool whose source code is known and which has been validated, to confirm the finding. Such corroboration, however, does not validate the search tool, only the result of the use of that tool at a particular time. To validate a proprietary tool generally using open source tools requires months of work, thousands of hours by highly experienced analysts, such as the FBI put in when validating EnCase. Of course, each time a new version of a tool comes out, more hours of validation are needed. Thus, while this means may be reliable, it is hardly practical.

A second means would be to use another proprietary tool -- say, Access Data's FTK -- to run the same search as EnCase performed and compare results. This method, however, is not truly scientific, since identical search results are just as likely to confirm that the two engines are identically flawed as they are reliable.

A third means to validate a proprietary search engine without revealing its code would be to search test data sets with known test results and which contain the types of data that the engine would search when regularly deployed. Comparing the results of the searches by the proprietary search engine to the known results should validate or invalidate the search engine. Again, however, such testing is extremely time-consuming and expensive. The designer would have to engage in such testing and publish its results; one could hardly expect the typical user of the search engine to engage in such studies.

As previously stated, the second Daubert hearing issue is where the searching was done. Specifically, the issue would be whether only the files actively stored on a hard drive, for example, were searched, or whether deleted files, temporary files or file fragments in the "unallocated space" of a hard drive were also searched. When ESI is gathered, unless bit stream, forensic images (i.e., exact copies of every 1 and 0 on a piece of media) are made, the deleted files, etc., will not even be present to search. To search for such ESI, forensic tools must be used. Thus, in United States v. O'Keefe, for example, the defendant challenged the government's search results for potentially exculpatory evidence in its possession by arguing that by not looking "everywhere" on the drive for deleted files or file fragments, the government had not fully discharged its duty to search everywhere.

The problem with searching "everywhere," however, is not so much a Rule 702 problem as a practical one: forensic searches of every possible file fragment take impossibly long, and if many hard drives and servers are involved, the impossible becomes unthinkable. O'Keefe, however, raises another issue, one far more interesting and conceptually difficult: for search engines, passing the Daubert test may depend upon whether one is trying to prove that something is there or that something is not there.

EVIDENCE AND ITS ABSENCE
Anyone who remembers examining scientific method when taking high school or college science classes will recall the question whether the absence of evidence that "x" is present means that "x" truly is not present or whether the test for finding "x" was simply insufficient. For example, while a PET Scan's positive finding for cancer is conclusive, a failure to detect cancer may mean the absence of cancer or that the PET Scan failed to detect cancer that was present.

Thus, the acceptance of a test as scientific proof under Daubert and Rule 702 is more likely when the test is to prove that something is present than absent. In Sanders v. Texas, 191 S.W.3d 272 (Ct. App. 2006), for example, the Texas Court of Appeals had no trouble affirming the trial court's findings that the expert's use of EnCase to create a bit stream, forensic image of the defendant's hard drive and his search of the drive to uncover child pornography -- both positive findings -- was scientifically valid. Since Encase's findings can be corroborated by a tool other than the proprietary one used, the validity of the imaging and search is much easier to establish.
In O'Keefe, Equity Analytics v. Lundin and the prototypical e-discovery matter, the typical challenge is the opposite of the typical challenge in a criminal matter: the requesting party's typical challenge to e-discovery production is not that it is inauthentic but that it is incomplete. The Daubert challenge in e-discovery cases is to prove that the search results yielded "everything."

If the search engine in question were an open-source tool, the challenge could be more easily met: the search engine's methodology would be open for all to test, and it would either work when searching test sets with known results or produce anomalies or mistakes. However, if the search engine is proprietary, proving the negative (it did not miss anything) by proving the positive (this is how it searches) is not available to the tool's proponent. The "third means" discussed above -- subjecting the search tool to known test data to see whether it missed any "hits" -- could work, but that means is extremely time-consuming, expensive and beyond the capability of the typical user.

THE MYTH OF PERFECTION
The Sedona Conference commentary provides an interesting method of "corroboration." It cites a study in which review attorneys, doing a "manual" review of discovery, were asked how much responsive data they were able to find. The attorneys guessed 75 percent, but a detailed analysis revealed that they had found only 20 percent. Using that study to illustrate what the Sedona Conference's commentary refers to as the "myth of perfection," i.e. that review attorneys slogging through e-documents and e-mails will catch responsive ESI that concept search engines will miss, the commentary makes the scientifically questionable but legally valid point that the validity of concept search tools must be determined by measuring concept search results against the actual results of review attorneys, not against results of a "perfect" search. If concept searching improves upon review practice as it now stands, it is a valid litigation tool.
In making its point, the Sedona Conference commentary returns to a touchstone of discovery practice: that when producing discovery, a "perfect review ... of information is not possible ... . The governing legal principles and best practices do not require perfection in making disclosures or in responding to discovery requests."

The Daubert challenge raised by Facciola, then, may be met not by judging the scientific validity of a search engine in an absolute way, but by judging how valid it is to suit the purposes of e-discovery production, an undertaking which involves many factors, such as the costs in time, money and energy to the producing party and their marginal benefit to the requesting party and the litigation, that have no bearing on the scientific validity of the search engine. In other words, the ultimate acceptance of an e-discovery search tool may be informed by its relative perfection but will ultimately depend, like so many other things in the law, upon the totality of circumstances.

Trial Solutions Offers SaaS Hosting for 5 Gigabytes for 5 Months for 5 Users for $5

noreply@blogger.com (Charles Skamser) — Wed, 11 Jun 2008 19:39:00 -0500

As a rule, I don't often talk about Trial Solutions or its products and offerings. However, I believe that their decision to offer ImageDepot, its SaaS based Hosting and Review Platform for $1 per Gigabyte represents a true paradigm shift in the eDiscovery market and therefore warrants mention on my Blog.

Trial Solutions, which has been very quietly offering ImageDepot for a number of years to its Service Provider Alliance Network, decided to begin distributing ImageDepot earlier this year to the eDiscovery market as part of its Litigation Support Ecosystem. With this aggressive new pricing, ImageDepot, the only true SaaS based production quality ORT in the industry based upon the Microsoft Technology Stack, could very quickly become an industry standard for online review. Or, at the very least will provide a very attractive alternative to any law firm or corporate legal department that is currently paying thousands of dollars per month hosting with one of the legacy hosting platforms.

The Company reports that the new ImageDepot promotion will enable users to host and review 5 Gigabytes of data for 5 months for 5 users for $5. Or, an unlimited amount of data for 5 months for an unlimited number of users for $5 with a 12 month hosting committment. This second option is available for both new projects and for transferring existing projects from another hosting platform. The Company notes that at the end of the initial promotional term that standard monthly hosting rates will apply. Further, the company indicated that the 5 for 5 Promotional offer will only be available through authorized ImageDepot Resellers.

For more information, you can contact Trial Solutions at 877-595-6464 or visit the ImageDepot Website at http://www.imagedepot.com/.

Interesting eDiscovery Technology Acquisition Opportunity

noreply@blogger.com (Charles Skamser) — Tue, 10 Jun 2008 16:22:00 -0500

This entry marks that first time that I have posted information about a technology vendor that has eDiscovery technology and the desire to be acquired. It is important to point out that I am not a broker and I am only posting this information as a source of information for the eDiscovery market.

The eDiscovery technology company believes they would be a good acquisition target for an email/data archiving company or a national litigation support/e-discovery provider that would like to have its own e-discovery processing/production and review platform. Currently, they are open to acquisition by a larger company that would provide sales and support infrastructure or a strategic investment that would allow them to build out their own infrastructure. The technology fits into the Processing, Review, Analysis and Production phases of the Electronic Discovery Reference Model (EDRM).

For more information about this company and transaction please go to the Litigation Support Industry: Business News and Information Blog.

Rule 26(b)(5) Clawback Caselaw Emerging

noreply@blogger.com (Charles Skamser) — Wed, 04 Jun 2008 22:42:00 -0500

Given the enormous increase in the amount of data encumbant in todays discovery, Federal Rule of Civil Procedure 26(b)(5) includes a section covering the accidental production of privileged information. It permits organizations to retract (“clawback”) privileged information following its discovery. Potential concerns regarding privilege are supposed to be discussed during meet-and-confer sessions as part of the discovery plan.

If information deemed privileged is inadvertantly produced, the producing party can notify the recipient of this assertion and the basis for it. The requesting party must return, sequester, or destroy the information promptly, and is barred from disclosing it until the claim is resolved.

Further, if the recipient has already disclosed the privileged information prior to notification, it must take reasonable steps to retrieve it. In any event, the producing party must preserve the privileged information until the litigation is resolved. Recipients can naturally dispute claims of privilege by submitting the information to the court for a ruling. The producer must make responsible attempts to avoid such disclosures—sloppy production is not an excuse and may imply a waiver of privilege. Moreover, producers must assert claims of privilege within “reasonable time,” again requiring a handle on the data. Supposedly, courts will weigh these factors in determining whether to waive or forfeit a claim of privilege.

Given all of this as the foundation, it appears that caselaw will determine the actual parameters and application of Rule 26(b)(5).

New Federal Ruling on Clawbacks

A federal magistrate in Baltimore has ruled a company sued for infringement has no attorney-client privilege in 165 documents mistakenly turned over to its opponent in e-discovery.

The documents included several communications between the company Creative Pipe Inc. and its lawyers.

U.S. Chief Magistrate Judge Paul Grimm ruled Creative Pipe had failed to demonstrate it took reasonable precautions to prevent disclosure when it used an untested keyword search to find potentially privileged documents. Lawyers told the Maryland Daily Record the opinion provides a cautionary tale to litigants.

“All keyword searches are not created equal,” Grimm wrote in the May 29 opinion (PDF). “Any order issued now by the court to attempt to redress these disclosures would be the equivalent of closing the barn door after the animals have already run away.”

Creative Pipe had requested a clawback agreement entitling it to retain the privilege on documents inadvertently disclosed, but abandoned the request when it was allowed four additional months for discovery. The company said the time would make it possible to conduct a document-by-document privilege review, according to the opinion.

Creative Pipe’s e-discovery expert had used about 70 keywords to find documents submitted to the company’s former lawyers for a privilege review. Documents that were not text-searchable were submitted to the lawyers for individualized review, but because of time constraints, the lawyers only looked at the document titles, the opinion says. Those that appeared to raise privilege issues were then read individually. At the time, the company was represented by lawyers from Meyer, Klipper & Mohr.

Grimm wrote that the defendants were “regrettably vague” in their description of the 70 keywords and said a growing body of literature highlights the risks of unreliable keyword review. The defendants did not sample the documents produced, “the only prudent way to test the reliability of the keyword search,” Grimm wrote.

He noted that courts have used three approaches to determine waiver when e-discovery documents are inadvertently disclosed, and said Creative Pipe fails both the strictest and an intermediate test that balances a number of factors.

The intermediate test considers the reasonableness of the precautions taken to prevent disclosure, the extent of inadvertent disclosures, any delay rectifying disclosure, and the overriding interest of justice.

“In this case, the defendants have failed to demonstrate that the keyword search they performed on the text-searchable [electronic evidence] was reasonable,” Grimm wrote. “Defendants neither identified the keywords selected nor the qualifications of the persons who selected them to design a proper search; they failed to demonstrate that there was quality-assurance testing; and when their production was challenged by the plaintiff, they failed to carry their burden of explaining what they had done and why it was sufficient.”

Grimm also noted this was not a case of one document slipping through the cracks, and the privileged documents that had been disclosed included e-mail communications between the defendants and their lawyers. He also said it was noteworthy that it was the plaintiffs that brought the disclosure to the court’s attention.

Jason Baron, director of litigation at the National Archives, told the Daily Record that “what Judge Grimm has done is give a roadmap to lawyers in the United States on how to present to a court how they went about searching for relevant documents.” Baron was the editor for a Sedona Conference Best Practices Commentary on e-discovery that was cited in Grimm’s opinion.

Proactive eDiscovery should be at the top of Enterprise Hierarchy of Needs

noreply@blogger.com (Charles Skamser) — Wed, 28 May 2008 23:29:00 -0500

It appears that the debate over the importance of proactive eDiscovery isn't going to linger in perpetuity like the 60 year debate over how Maslow's Hierarchy of Needs pertains to the Enterprise or the other well know corporate decision about whether to stock Coke or Pepsi.

According to The Radicati Group, the total email archiving market (including both on-premises archiving solutions and hosted archiving services) reached almost $1.3 billion by the end of 2007, and should grow to over $6 billion by 2010. Other analysts such as Gartner and Forrester show similar trends.

However, there are still some in the industry that are contending that the enterprise still has time to investigate and move forward at a more measured approach based upon the potential risks (e.g. sanctions and potential losses due to the costs associated with reactive eDiscovery).

As an example, in an article titled "Driving Toward Cost-Effective E-Discovery", by Jeffrey Beard, published in the May 2008 issue of InsideTech, Mr. Beard contends that well thought out processes and procedures may be a sufficient remedy in the short term. And, although I agree 100% that top down driven enterprise wide process and procedure are a necessary component for success, proactive eDiscovery technology needs to be at the top of every enterprise list for implementation in the 2008-2009 time frame. With costs plummeting and sophistication increasing at an accelerating rate for email archiving, enterprise wide Electronically Stored Information ("ESI") achieving, off-site/online storage and all of the associated Litigation Readiness eDiscovery infrastructure, there is no longer any financial justification for any enterprise at any level to not have some type of proactive solution already implemented.

And, in light of the afforable proactive eDiscovery options, I am beginning to see the courts and juries less and less tolerant of enterprises contending that the costs of reactive eDiscovery is prohibitively high or too complex. As an example, Medtronics, having already invested millions just to retrieve and review documents, including vast amounts stored on backup tape, was stunned by a jury verdict of $570 million and learned a hard lesson about retaining too much electronic information. Morgan Stanley was ordered to pay $604 million in damages to an investor after it failed to produce e-mails in a fraud case, and Phillip Morris was ordered to pay court-imposed sanctions of $2.9 million for failing to comply with its own document retention policy.

Proactive eDiscovery is obviously not going to create the same debate as Maslow's Hierarchy of Needs. Nevertheless, it is always interesting to watch the enterprise drag its feet or reluctantly "Cross the Chasm" even in the face of overwhelming evidence that it must do so to just keep pace with the pack or worse yet face financial ruin.

The full content of Mr. Beard's thought provoking article titled "Driving Toward Cost-Effective E-Discovery" is as follows:

In the first 18 months after the enactment of the amended Federal Rules of Civil Procedure (FRCP), companies tend to be at various stages in their internal initiatives and overall litigation readiness. Some litigation-heavy industries have felt the sting more strongly, including pharma, health care, energy, insurance and financial, so many of the larger players in these markets have already established in-house litigation readiness programs, processes and procedures. Others have experienced several substantial matters involving electronically stored information, not as a routine, but more akin to intermittent fire drills upon which to test their readiness mettle.

When everything is added up, though, reactionary measures often cost more than proactive ones. Why? Because reactionary processes are at best ad hoc, generated on-the-fly as the need and urgency compels decision-makers at all levels, from the GC and CIO down to the paralegals and IT personnel charged with their execution. As we’ve seen in various reported cases, they introduce huge inefficiencies and costly mistakes, particularly in the areas of preservation, collection, and the timely and effective imposition of an internal legal hold. Also, not having a smoothly practiced and executed preservation/collection plan causes additional and very substantial costs to be incurred downstream, particularly during the most costly phase—review.

Those looking to more proactively increase their readiness have increasingly begun to turn to e-mail, content and records management. However, unless it’s a bet-the-company or other high-profile case, a floodgate-opening precedent or the promise of a stinging sanction, it often doesn’t make sense to write a blank check on e-discovery. The new rules introduced the concept of proportionality, balancing the costs of discovery with the potential monetary amount at issue. Indeed FRCP Rule 1, sometimes referred to as the mission statement of the new rules, tells us they “should be construed and administered to secure the just, speedy, and inexpensive determination of every action and proceeding.”

Inexpensive? The harsh reality is that the legal market isn’t there yet, not by a long shot. So where does it make sense to invest in litigation readiness and ultimately reduce your risks and associated costs? Naturally, it depends on your litigation and e-discovery risk, your company’s tolerance for those risks and the financial and human resources that can be applied. In-house legal departments tend to lack in one or both of the latter due to the regular push of business and financial pressures to keep head count reasonable.

Many companies already have defined processes in varying stages of development and execution, so the next logical step is to assess their inherent strengths and weaknesses—what has worked well and what hasn’t. From that analysis, you can determine your ongoing inherent risk as well as areas for improvement and additional investment. However, it can be challenging for in-house groups to have a truly objective perspective of their internal processes. Sometimes they are simply too close to the problem or, worse, are part of it, whether they realize it or not.

For example, where are the “black holes” in your internal collection efforts? Sometimes key things get overlooked when the professionals tasked with these efforts have conflicting priorities and workloads. These problems are further compounded by miscommunication and changing expectations due to the matrix nature of cross-functional groups.

This is where it often pays dividends to bring in the broader perspective of an outside litigation readiness consultant who will take a more objective and comprehensive approach. Organizations can then use their recommendations to address the highest risk areas first, coupled with best practices derived from working with similarly situated organizations.

E-mail still remains one of the highest risk areas, and worthy of further efforts. While tools exist to help with collection, review and early assessment, those tools are most effective when you have implemented effective processes and procedures within your organization. For instance, do you have policies which allow the creation of separate e-mail archives on each user’s local computer hard drive? These are often referred to as PSTs for Microsoft Outlook and NSFs for Lotus Notes. Having these additional custodian repositories can vastly drive up the cost and complexity of your litigation readiness and response to requests, with additional preservation, collection, deduplication, search and review requirements. Do you have consistent auto-deletion periods for all custodians or defined classes of custodians? As e-mails can also become business records, are your e-mail system policies tied to your records management policies? The more consistency you have across your systems and policies, including in their execution by users and administrators alike, generally the better your overall defensibility.

Recent case law addressing e-discovery-related issues suggests that among the greatest risk areas are the timely and effective imposition of a litigation hold and the resulting successful identification, preservation and collection of all relevant e-mail and other data. A number of organizations are still doing legal hold notices the “old fashioned way”—e-mailing notices to identified or potential custodians.

While leading e-mail systems allow for sending delivery and/or read receipts from the intended recipients, further automation support is likely required to consistently address tasks such as custodian follow-up and tracking, auditing and reporting the statuses for interviews, collections and ongoing preservation efforts. There are a growing number of third-party systems which address some or all of these needs, while some companies have turned to their internal technology professionals to build them instead. This is a key area in which to involve a qualified litigation readiness consultant, who can help identify your specific needs, assimilate the requirements for inclusion in RFIs and RFPs and work with outside system providers and your IT professionals to select the solution best matched to those needs.

For organizations looking for better ways to achieve their goals with limited resources, these are key steps that can lead to meaningful improvements and substantial reductions in their overall risk and cost. The important thing to keep in mind is that the solution is not a product, but a well-designed and defined process supported by good documentation, engaged and motivated people, and training, practice and automation where needed.

In Search of Integrated Conceptual eDiscovery Search Technology

noreply@blogger.com (Charles Skamser) — Sat, 10 May 2008 09:23:00 -0500

Over the past 6 months I have been investigating cost effective, integrated conceptual eDiscovery search technology delivered under a SaaS model. The basis for this investigation is to identify a way to extend the current capabilities of eDiscovery search through a forward thinking search technology that can be tightly integrated on the same Microsoft stack based eDiscovery platform with email archiving and other proactive data retention technology, Electronic Data Discovery (EDD) software and an Online Review Tool (ORT). My finding are that the current state of forward thinking search technology is such that it requires the support of a separate and proprietary database and therefore does not lend itself to integration with EDD and ORT platforms that sit on standard SQLServer solutions.

Where this current state of the market leaves the user is with a choice of either moving large amounts of data or least large amounts of index files and associated data between platforms or investing in a completing propriety eDiscovery solution.

In the process of this investigation, I have found several outstanding articles that touch on the various topics incumbent in this discussion. The first article, found on Law.com, titled "In Search of Better E-Discovery Methods" by H. Christopher Boehning and Daniel J. Toal, does an excellent job of discussing some of the standard criteria for new search technology and whether or not it surpasses currently available keyword and Boolean search technology.

The second article is actual a Blog posting by Cher Devey, titled "Alternative Search Technologies - Too Good to be True" on her "eDiscovery Myth or Reality?" Blog. Ms. Devey discusses the concept and viability of human intervention into the search process. (Please note that the full text of Ms. Devey's Blog Post can be found at the bottom of this posting).

The full text of Mr. Boehning's and Mr. Toal's article is as follows:

As the burdens of e-discovery continue to mount, the search for a technological solution has only intensified. The holy grail here is a search methodology that will enable litigants to identify potentially relevant electronic documents reliably and efficiently.

In an effort to achieve these often competing objectives, litigants most commonly search repositories of electronic data for documents containing any number of defined search terms (keyword searches) or search terms appearing in a specified relation to one another (Boolean searches). These search technologies have been in use for years, both in litigation and elsewhere, and accordingly are well understood and widely accepted by courts and practitioners.

But keyword and Boolean searches are far from perfect solutions; they are blunt instruments. Such searches will identify only those electronic documents containing the precise terms specified. These methodologies therefore will not catch documents using words that are close, but not identical, to the specified search terms, such as abbreviations, synonyms, nicknames, initials and misspelled words.

On the other hand, using more search terms may reduce the risk that an electronic search will miss a relevant document, but only at the price of increasing -- often quite dramatically -- the number of irrelevant documents found in the search. This is a serious problem because counsel must manually review whatever documents the searches yield in order to sift out non responsive materials, make privilege determinations and designate confidential documents. Keyword and Boolean searches thus require a careful balance to be struck: Unduly restrictive searches may miss too many responsive documents while over broad searches threaten stratospheric discovery costs.

Against this backdrop, courts and litigants understandably have been intrigued by the claims of those promoting alternative search technologies, such as "concept searching." The vendors of such technologies suggest their search strategies are able to identify the overwhelming majority of responsive documents while virtually eliminating the need for lawyer involvement in the review process.

Such claims strike many in the legal community as too good to be true. And their skepticism is appropriately heightened because the precise methodologies that such vendors use often are shrouded in mystery, owing to their stated desire to safeguard their proprietary processes and techniques. But this also means their tantalizing claims cannot readily be subjected to independent scrutiny. The question thus posed -- and still largely unexplored -- is whether these alternative search technologies have anything to offer and, if so, how best to evaluate the competing technologies and the often sensational claims of their promoters.

To evaluate whether an alternative search technology might be helpfully employed in any particular case, it is first essential to understand how it works. Some of the principal alternative search technologies, which fall under the broad heading of "concept searching" methodologies, are as follows:

Clustering. Whereas keyword and Boolean searches mechanically apply certain logical rules to identify potentially relevant documents, clustering relies on statistical relationships, which results in documents containing similar words being clustered together in relevant categories. The clustering tool compares each document in a pool to "seed" documents, which have already been designated as relevant. The more words a document has in common with a seed document, the more likely it is to be about the same subject and therefore to be responsive. Moreover, clustering tools generally rank documents based on their statistical similarity to the seed documents.

Taxonomies and ontologies. A taxonomy tool is used to categorize documents containing words that are subsets of the topics relevant to a litigation. For example, if one of the topics of interest is "dogs," a taxonomy tool would capture documents that mention "golden retrievers," "poodles" and "chihuahuas." Ontology tools perform similar searches, but are not confined to identifying subset relationships. Building on the last example, an ontology tool would capture documents that mention "kennels" or "veterinarians."

Bayesian Classifiers. Bayesian search systems use probability theory to make educated inferences about the relevance of documents based on the system's prior experience in identifying relevant documents in the particular litigation. The search results then would be ranked based on the predicted likelihood of their relevance to the litigation.

HOW APPROACHES COMPARE
These alternative search technologies may sound promising in concept, and the claims about their efficiency and accuracy likely add to their allure, but the question remains whether these approaches outperform the standard search approach.

Keyword searching (including with the use of Boolean connectors), its acknowledged limitations notwithstanding, has secured such widespread acceptance for a reason. As an initial matter, the technology and search methodology is well understood and familiar to anyone who has used Westlaw, Lexis or similar search engines. It therefore can be easily discussed with both opposing counsel and judges. The simplicity of keyword searching also doubtlessly promotes negotiated resolution of discovery disputes because the parties have less reason to fear that ignorance about the technology will lead them to strike a bad bargain.

But the simplicity of keyword searching is also its principal weakness. Keyword searches capture only documents containing the precise terms designated, which virtually assures that such a search will miss relevant documents. And, on the other side of the equation, keyword searches will mechanically capture every document -- whether relevant or not -- containing any search term. This means keyword searches may be both substantially under- and over-inclusive. Concept searching systems, by contrast, are not dependent on a particular term appearing in a document and therefore may locate documents a Boolean search would not. But they may suffer from other infirmities.

So how does concept searching stack up? The best evidence to date comes from the Text REtrieval Conference, which in 2006 designed an independent research project to compare the efficacy of various search methods. In view of the prevalence of keyword and Boolean searches in litigation today, TREC was particularly interested in determining whether the alternative search methodologies outlined above were better than Boolean.

As its starting point, the TREC study used a test set of 7 million documents that had been made available to the public pursuant to a Master Settlement Agreement between tobacco companies and several state attorneys general. Attorneys assisting in the study then drafted five test complaints and 43 sample document requests (referred to as topics). The topic creator and a TREC coordinator then took on the roles of the requesting and responding counsel and negotiated over the form of a Boolean search to be run for each document request.

In addition to the Boolean searches, computer scientists from academia and other institutions attempted to locate responsive documents for each topic utilizing 31 different automated search methodologies, including concept searching. The results were striking. On average, across all the topics, the negotiated Boolean searches located 57 percent of the known relevant documents.

But none of the alternative search methodologies reliably performed any better. That is to say, for each topic, the Boolean search did about as well as the best alternative search methodology.

Interestingly, although the Boolean searches generally outperformed the alternative search protocols, the methods did not necessarily retrieve the same responsive documents. In fact, when all of the responsive documents found by the 31 alternative runs were combined, TREC discovered that the alternative search runs collectively had located, on average, an additional 32 percent of the responsive documents in each topic.

As a result, while the Boolean search generally equaled or outperformed any of the individual alternative search methods, those searches also captured at least some responsive documents that the Boolean search had missed.

COST ANALYSIS
This suggests that even if alternative search methodologies have not yet been shown to beat Boolean searches, their use to supplement Boolean searches might increase the number of responsive documents located. But at what cost? The potential benefits of locating any additional documents through use of an alternative search methodology would still have to be weighed against the cost, both in money and resources, required to locate them.

The relevant cost here is not just the price of using the alternative search technology, but also the number of false positives identified by the approach (i.e. documents retrieved by the search, but turn out not to be responsive). Any automated search method -- whether a keyword or concept search -- will yield false positives, which counsel must review and filter out prior to production, which can be a costly process. It therefore is far from clear that use of an alternative search methodology in addition to a keyword or Boolean search will be appropriate in any particular case, a question the TREC study does not attempt to address.

For now, the available evidence suggests that keyword and Boolean searches remain the state-of-the-art and the most appropriate search technology for most cases. This seems particularly true when keyword or Boolean searches are used in an iterative manner, where litigants: (i) negotiate search terms and Boolean operators, (ii) run the agreed-upon searches, (iii) review the preliminary results, and (iv) adjust the searches through a series of meet-and-confers. This type of "virtuous cycle of iterative feedback" has been endorsed by courts and commentators alike.

The intuition of the legal community that an iterative approach to electronic discovery promotes reliability and efficiency finds empirical support in the TREC study. As part of its study, TREC employed an expert tobacco document searcher who used an "interactive" search methodology.

TREC found that the expert searcher located, on average, an additional 11 percent of the relevant documents beyond those that had been located by the initial Boolean searches, which means that an interactive Boolean approach ultimately located 68 percent of the relevant documents -- far better than any of the alternative search methodologies.

CONCLUSION
It may be that alternative search methodologies eventually will surpass the performance of keyword and Boolean searches, but that day does not yet seem to have arrived.

The independent research conducted to date suggests that, for the time being at least, nothing beats Boolean, particularly when used as part of an iterative process.

That does not necessarily mean that alternative search technologies are not worth considering, either independently or along with Boolean or keyword searches. But practitioners would be well advised to carefully scrutinize the marketing claims of the purveyors of such technologies and to factor in often substantial direct and indirect costs of such approaches.

H. Christopher Boehning and Daniel J. Toal are litigation partners at Paul, Weiss, Rifkind, Wharton & Garrison. Associate Jason D. Jones and Aaron Gardner, the firm's discovery process manager, assisted in the preparation of this article.

The Full Text of Ms. Devey's Blog Posting is as follows:

It seems that alternative search technologies (alternative to the familiar Keyword and Boolean searches) touted by Vendors are considered as ‘too good to be true’. Check it out yourself at In Search of Better E-Discovery Methods By H. Christopher Boehning and Daniel J. Toal, New York Law Journal April 23, 2008

The above legal article also mentioned the Text Retrieval Conference (TREC) 2006 study which was also examined by Will Uppington in the article, Better Search for E-Discovery, March 11th, 2008

What I find interesting in Will Uppington’s article is the finding; ‘One of the best ways to get better search queries is to commit human resources to improving them, by putting a “human-in-the-loop” while performing searches’.

Reading in between these two ‘search themed’ titles, one from the legal side and the other from a technical perspective, highlighted the contrasting findings and interpretation on the TREC 2006 study

What else can we say/talk about the ‘human-in-the loop’, the ‘virtuous cycle of iterative feedback’ & “interactive” search methodology?

Well such phrases/concepts are not new. What is new is that the ‘human actions’ aspects are creeping (awareness?) into the ediscovery space. Other knowledge researchers outside the ediscovery domain have been busily coming up with phrases/concepts such as the ‘concept searching’ methodologies. Reality (or inertia adoption) testing of such newer technologies are clearly not well understood (too good to be true?) by the courts and practitioners.

On human actions and computer programs, a beautiful quote comes from my friend, Roger C: “While computer programs can write other computer programs, they can’t write the first program”.