Reasonable and Proportionate Methodologies Sufficient to Meet Discovery Obligations
Tremblay v. OpenAI, Inc., (N.D. Cal. July 31, 2024)
Why Is This Case Law Ruling Important?
In a recent ruling, the United States District Court for the Northern District of California emphasized the importance of using "reasonable and proportionate methodologies" in meeting discovery obligations, as a reminder to litigants that they must carefully consider the scope and methods of discovery requests in order to avoid excessive costs and burden.
E-Discovery Ruling Overview
In this class action copyright case against OpenAI on behalf of authors whose works were used to train defendant's large language model artificial intelligence software, plaintiffs and defendants disagreed on three significant areas surrounding the ESI protocol for discovery. These disputes underline the complexities involved in determining proportionate and effective methodologies for e-discovery validation protocols.
Disclosure of Search Terms
The parties disputed whether the producing party should disclose its search terms prior to the review process and whether the requesting party's input should influence the determination of those terms.
Evaluation Method for Validating Searches
A second area of disagreement concerned whether recall or elusion was the more appropriate methodology for evaluating the efficacy of the search process.
Confidence Metric for Sampling
Finally, the parties disagreed on the appropriate confidence interval and range for sampling, with plaintiffs proposing a confidence metric of 98% +/-2% and defendants advocating for 95% +/-5%.
E-Discovery Case Ruling
- Disclosure "Appropriate and Reasonable": Plaintiffs sought collaborative input in defining search terms, but defendants argued this was unnecessary and inconsistent with court ESI protocols. The court addressed the dispute over search terms, ruling that while the disclosure of search terms by the producing party is “appropriate and reasonable,” the requesting party should not have input into their determination. The judge agreed with defendants, noting that allowing such input “raises the specter of too many future delays and disputes over methodology and search term formulation.”
- "Reasonable and Appropriate Methodologies... Required": The court mandated that the ESI protocol include provisions for adopting reasonable and proportionate methodologies compliant with legal standards for identifying, searching, collecting, culling, reviewing, and producing ESI. The protocol must account for varying methodologies across different data sets and require the parties to meet and confer in good faith to resolve any disputes related to their ESI productions.
- "Meet and Confers" a Tool for Dispute Resolution: The court resolved the parties’ dispute over validation procedures by mandating specific measures in the ESI protocol. He required parties to disclose their methods for evaluating search efficiency and identifying confidence metrics for sampling. To ensure compliance with legal standards and effective discovery, both parties must "take reasonable steps to validate [their] review process," including using quality control measures to identify missing relevant ESI or excessive irrelevant ESI. The protocol allows reasonable requests for validation details, including disclosure of end-to-end recall levels, while disputes over validation should be addressed through good-faith meet-and-confer discussions.
ESI protocols originally were party agreements as to form of production and other nuts and bolts of the production. As the scope of protocols have expanded, we are seeing more situations where the parties submit disagreements as to the language and scope of the ESI Protocol to the court for resolution. Judges disagree about whether that is an appropriate role for the Court. At times, the judge can do little more than decide easy issues (here, produce the search terms but opposing counsel has no role in the terms) and otherwise require counsel to meet and confer when disputes arise down the road.Hon. Andrew Peck (ret.), Senior Counsel, DLA Piper
Case Law Tip
The courts don’t expect e-discovery professionals to be perfect, but they do expect collaboration and cooperation. Review Exterro’s guide to the FRCP if you’d like a refresher.
