There’s good news in the world of electronic discovery. This February in New York, Magistrate Judge Andrew Peck and counsel for the parties in Da Silva Moore v. Publicis Groupe gave us a magnificent e-discovery lesson and pushed open the door for the utilization of advanced search technologies -- namely predictive coding, an increasingly used methodology of computer-assisted review.
The plaintiff filed a Title VII class action gender discrimination claim against defendant Publicis Groupe, alleging she and other female employees at Publicis Groupe endured discriminatory terminations, demotions and job reassignments. The plaintiff (who had very little, if any, electronically stored information (ESI) of her own to produce) demanded that Publicis Groupe produce documents (including ESI) that related to whether Publicis Groupe:
- Compensated female employees less than comparably situated males through salaries, bonuses or perks.
- Precluded or delayed selection and promotion of females into higher-level jobs held by male employees.
- Disproportionately terminated or reassigned female employees when the company was reorganized in 2008.
Based on the records requested and the number of custodians, the parties anticipated the document pool would be around three million documents, which would have likely cost in excess of $1 million with traditional keyword search methods. Instead of going this route, the parties agreed to something bold: review the documents using what has come to be called predictive coding, a methodology of computer-assisted review. By using these methods, the parties hoped to reduce the number of manually-reviewed documents from 3 million to 20,000.
The implementation of predictive coding is not simple. Fortunately, Da Silva Moore v. Publicis Groupe provides a lengthy guide on important topics such as methods to identify the initial seed set, iterative training rounds to refine the “predictive coding to assure reasonable recall” and methods of sampling to validate levels of confidence and confidence intervals.
What the Case Means for Your Business
Although discovery in Publicis Groupe is far from over, and the parties have each filed motions challenging portions of Judge Peck’s ruling, there are already lessons to be learned for how to effectively deploy computer-assisted review to reduce the cost of electronic discovery in your cases:
1. Have an expert, knowledgeable about the review tool you intend to use. Judge Peck turned to the parties’ technical experts to explain the effect of the review protocol on the validity of the ultimate production. Surely judges less familiar with the technology could benefit from hearing from an expert in the field. Since experts tend to disagree (as they did in Publicis Groupe), it’s an absolute requirement to provide testimony about the operation and testing of the search tool chosen for the case.
2. Be willing to accept that you will not receive every potentially relevant document. Judge Peck put it best when he reminded counsel, “By the time you go to trial, even with six plaintiffs, if you have more than 100 trial exhibits it will be a miracle.” He also explained that, “The idea is not to make this perfect, it’s not going to be perfect. The idea is to make it significantly better than the alternative (human review) without nearly as much cost.”
Consequently, you have to be willing to risk that a computer will miss more documents than the recent law grad you would normally pay to sift through each page. Keep in mind that human review and key word search strings are far from perfect. Predictive coding when properly applied will likely enhance both recall and precision.
3. Cooperate with the opposition. The utilization of this technology requires engaged cooperation between the parties. Counsel must review and share the initial seed set with the opposition, and agree on statistical sampling techniques. Notwithstanding subsequent disputes, Da Silva Moore v. Publicis Groupe illustrates competent counsel working closely on e-discovery to meet the interests of both the plaintiffs and the defendant. Keep in mind that both sides agreed to utilize this advanced technology in this case. Of course, the devil is in the details, where reasonable litigants can disagree.
4. Understand the technology. Even with technology experts at the ready, counsel were still necessary to advocate for their client’s interest in balancing the cost of discovery against the completeness of the final set of documents produced. Predictive coding is not right for all cases. It is not inexpensive—counsel must expend considerable up-front fees identifying the seed set and fine tuning the technology.
Touted as a practical, cost-saving and revolutionary solution, computer-assisted review is finally getting its chance to show what it’s worth. The private bar is watching anxiously to see whether it lives up to its billing.
This article was originally published in Inside Counsel.