"Because Something Must Be Done": The Dangers of Trying to Save E-Discovery Costs by Treating Data Like Paper
Remember the days before the computer, the typewriter, the printing press, and even carbon paper? The days when, to copy a book, one needed to sit down and re-write it by hand? Every letter of every word of every sentence-dappled paragraph, had to be painstakingly copied one by one. With all the technology available today, no publisher would ever consider copying a book by hand rather than re-printing a copy saved on the computer. To do so would be a waste of time -- a return to the Dark Ages.
When it comes to e-discovery, unfortunately, the Dark Ages still occasionally guest stars in modern-day electronically stored information (ESI) retrieval and production. The dangers of being such an e-discovery ostrich were most recently highlighted in Multiven, Inc. v. Cisco Systems, 2010 WL 2813618 (N.D. Cal. July 9, 2010). Plaintiff Multiven, along with the two counterclaim defendants, originally elected to undertake a manual review of the full set of voluminous ESI it possessed prior to production. Sound conscientious? Not exactly. It means they turned the clock back about 15 years (in e-discovery time, akin to 150 years) and:
(1) Refused to use an outside vendor to help organize ESI information;
(2) Refused to utilize any search terms to narrow the "giant mass" of data to be reviewed;
(3) Instead used approximately five attorneys for six months to a year, to manually review every unfiltered page of "that giant mass" for responsive documents.
Why? They wanted to save on cost, particularly the expense of hiring an outside vendor to help whittle down potentially responsive information. Perhaps they adopted a manual review for other strategic reasons. (Never mind, of course, the hourly billing rate of five attorneys doing eight hours of document review five days a week for over six months, which more likely than not was more expensive than hiring a vendor to narrow the "giant mass" to a more reasonable review load for the attorneys). The end result, however, was not exactly what was intended . . .
What sounded like a Great Idea to Multiven ultimately backfired. The process bogged everyone down, taking months and months. Finally, the district court became distressed when it became apparent that the delays associated with the manual review of all electronic files was causing the case to extend well beyond the boundaries of the scheduling order:
[I]t has become clear to this Court that [the plaintiff and counterdefendants] cannot complete their review and production of documents with enough time before the close of discovery to allow Cisco time to actually do anything with them. [They] so far have insisted on a review process that guarantees that they will not finish this extensive project in any reasonable amount of time.
Id. at *2. The trial court thus adopted the Report and Recommendation of Magistrate Judge Lloyd, required Multiven to utilize a vendor to assist in e-discovery, and appointed a Special Master to manage certain e-discovery disputes that potentially threatened to derail the litigation. In short, the Court concluded (id. (emphasis added)):
Because something must be done, this Court recommends that the District Court order the parties to promptly retain a third party vendor to assist with this increasingly perilous situation.
So first, Multiven was not only stuck having to hire the very vendor it had hoped to avoid, but on top of it had to pay its attorneys the hundreds of thousands it cost to conduct the months of document review that were done to avoid hiring a vendor in the first place. Instead of saving money, their gamble multiplied the cost of e-discovery in the case. Second, by this point, even Multiven was so fed up with the document review process that it was amenable to hiring a vendor before the court order even issued. Sure, part of it was because the defendant, Cisco, had offered to pay for part of the vendor cost, but Multiven had finally learned that it was in over it head, and that in trying to save money, it had wound up spending significant more cost on e-discovery.
While the Court's intervention was necessary to get the case back on track, one wonders what possessed the Multiven and its counsel to undertake a prolonged manual review, especially when Cisco appeared to be urging Multiven to employ modern technology all along. Delay was only one by-product of Multiven's decision. A second by-product was the likely extraordinary costs such manual reviews entail. The most substantial e-discovery costs arise from the attorney review process, regardless of whether that review is done internally by firm lawyers or outsourced to vendors in the United States or abroad.
One might think that a manual review is the "gold standard" and the most thorough, comprehensive way of searching. In other words, while you can place fistfuls of hay at a time into a hay-sorter to separate out those few stray needles, there's still a margin of error. You're more likely to find them if you use the time-consuming method of picking up one strand at a time and eyeballing it. Right? Oddly, no. In truth, the so-called gold standard of manual review is mythological, and the human eye can miss a lot more than a well-crafted keyword search protocol can capture. A widely-regarded study, Blair et al., Wittgenstein, 'Language and Information: Back to the Rough Ground!' 302 (2006), found that manual review teams only found 20% of the relevant documents! Ralph Losey and Jason Baron have also demonstrated that adding conceptual search tools to key-word searching further improves results and reduces cost. So Multiven's Dark Ages strategy not only multiplied costs and delayed the proceedings, but likely would have missed most of the relevant documents -- leading to potential sanctions and do-overs. Some cost-saving strategy!
In truth, there are very limited circumstances in which a case or production might require every e-document to be reviewed: the all-consuming, unfiltered, manual document review is dead, and for good reason. Today's staggering volume of ESI makes such a manual review both cost and time prohibitive. Utilizing the broad array of tools to cull down, de-duplicate and search data volumes is generally accepted and is even becoming mandated practice, as Multiven learned too late. While all searches are not created equal, the Text REtrieval Conference (TREC) Project -- co-sponsored by the National Institute of Standards and Technology (NIST) and U.S. Department of Defense-- has demonstrated a better search results by machines than plowing through data manually. While a search methodology utilized by a party has to be defensible, standards are slowly emerging, as documented by The Sedona Conference® in its May 2009 Commentary on Achieving Quality in the E-Discovery Process: Best Practices for Document Retention and Production. There is no reason to be stuck in the Dark Ages of linear, manual review.