The "Wake-Up Call" Rings Twice: Blawgs Buzzing on the Importance of Thoughtful, Collaborative Search Protocol Design

In reviewing the E-Discovery blawgs this week, a recurring theme emerged:  the pitfalls and limitations of keyword searching, and the need for collaboration and cooperation between counsel to devise effective search protocols.

A number of the E-Discovery blawgs this week featured the ABA Journal's article on improving e-discovery search protocols: "In Search of the Perfect Search."  The thought-provoking article highlights the dismaying lack of progress in finding cost-effective ways to locate and retrieve relevant documents through keyword searching. Bottom line? The way we are doing it now is wrong. Although technology has advanced at a rapid clip over the past twenty years, it has not been able to overcome "the fundamental ambiguity of language": research shows that paralegals and attorneys using simple keyword searches on a group of documents find, on average, only twenty percent of the relevant documents. That is basically the same result research found in similar studies conducted twenty years ago. 

The answer? Using a combination of search methods and tools. The Text Retrieval Conference Legal Track is working on a process and protocol to improve the results of digital searches. According to the article, however, few E-Discovery vendors have participated in the project, perhaps reluctant to have the effectiveness of their own search technologies quantified. Early recommendations from the project on how to improve on the usual keyword search include:

 

  • Working with opposing counsel to identify the best search terms;
  • Negotiating proposed Boolean search strings;
  • Using sampling to see whether the search engines are really finding the relevant documents.

Perhaps not-so-coincidentally, the April 3 post from the Electronic Discovery Blog discusses a Southern District of New York opinion that emphasizes these same points in delivering a "wake-up call" to the local bar "about the need for careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms or 'keywords' to be used to produce emails or other electronically stored information . . . " In that case, a dispute over delays and defects in a Bronx Courthouse construction project, parties sought emails from non-party Hill, the construction management company. One party proposed searching solely with the names of the parties and the names of the project as search terms. The other parties all proposed very broad keywords that would result in production of the entire database. Non-party Hill failed to contribute any helpful suggestions. 

 

As a result, the court was forced to create a search protocol itself, without meaningful assistance from the parties. The court closed its "wake-up call" with some golden words of advice for attorneys practicing in the SDNY, which bear repeating for attorneys and businesses facing e-discovery issues across the country:

 

Electronic discovery requires cooperation between opposing counsel and transparency in all aspects of preservation and production of ESI. Moreover, where counsel are using keyword searches for retrieval of ESI, they at a minimum must carefully craft the appropriate keywords, with input from the ESI’s custodians as to the words and abbreviations they use, and the proposed methodology must be quality control tested to assure accuracy in retrieval and elimination of “false positives.” It is time that the Bar — even those lawyers who did not come of age in the computer era — understand this.

 

Words to live by.