10 Jan 2013

“Barney It Down”: Reviewing the Basics of Predictive Technologies within E-Discovery

Posted in: Thursday E-Discovery Case Law No Comments

By Scott Giordano, Esq.

Exterro and Williams Mullen held a second roundtable discussion Wednesday in Washington D.C. about the practical and ethical aspects of predictive technologies. Building upon the insight gained at Tuesday’s New York event, a diverse group of attendees (inside counsel, government agencies, law firm counsel, and e-discovery service providers) received useful insights into the intricacies of predictive technologies from a panel of e-discovery experts, which consisted of FTC litigation support manager Chad Papenfuss and Williams Mullen partners Bennett Borden and Jay Brudz. To facilitate a lively discussion between the attendees, the CLE event centered around a “ripped from the headlines” hypothetical concerning a multi-national corporation involved in False Claim Act litigation (with nearly two terabytes of electronically stored information (ESI) and 80 custodians) and the role predictive technologies could play within e-discovery by the responding corporate legal team.

Key Takeaways from the Discussion:

Make the introduction of predictive technologies as simple as possible. An idea gleaned from the event was to “Barney it down” or make the process so simple that Barney the purple dinosaur could explain it (demonstrating once again the efficacy of a (really) “back to basics” approach). The biggest challenge facing counsel concerning predictive technologies seems to be adequately understanding the technical/statistical elements of predictive methodologies such as f-scores, elusion rates, etc. Without such an understanding, it is difficult to appreciate just how effective predictive technologies can be within e-discovery. It’s easy for e-discovery practitioners to forget that the vast majority of end users of e-discovery technology have not waded into the deep end of the predictive pool and there remains much work in educating them on the potential of this fusion of machine learning and human expertise.

Human linear review shouldn’t be considered the gold standard. A key takeaway articulated by Borden was the idea that from the time that the Federal Rules were established until recently, human linear review was considered the gold standard without anyone ever conducting the kind of testing—proving the null set, etc.—performed routinely with predictive technologies. No doubt most litigators would cringe at the thought of going back and testing past all-manual-review productions against predictive and seeing just how many important documents were missed (and at what cost). But when such tests have been conducted, Borden indicated that the results are indeed cringe worthy, finding that numerous responsive documents are missed during human linear review. Moreover, he described the frustration of explaining to the judge the concept of margin of error for missing responsive documents, and having the judge reply, “why don’t you just go back and get all of those missed documents?”  Again, much work in educating remains.

Predictive technologies are an efficient and effective way to rapidly satisfy document requests. The panel agreed that predictive technologies offers corporate legal teams the ability to quickly and thoroughly respond to government requests before they become formal subpoenas or civil investigative demands (at which time publically-traded companies have to report them in their SEC filings).  Similarly, predictive technologies can easily be re-purposed for civil litigation where the client is interested in resolving the matter as quickly as possible for the same reason. In both cases, employing rolling document productions that make use of ESI that is readily available is key, and predictive is very amenable to analyzing waves of additional documents as they become available.


Much like at the previous predictive technologies CLE in New York, the audience expressed an abundance of enthusiasm for utilizing predictive technologies within e-discovery. Members fired off question after question to the panelists and the panelists responded with excellent insight into the need to know elements concerning predictive technologies. This enthusiasm is consistent with other leading-edge e-discovery events such as the one held in Georgetown last December.

Much like on Tuesday, the event proved to be very informative, and no doubt we look forward to hosting many more to come.

To learn more about Exterro’s predictive technology capabilities click here.


Scott Giordano is an experienced attorney with more than 16 years legal, technology and risk management consulting experience. Holding both Information Security Systems Professional (CISSP) and Certified Information Privacy Professional (CIPP) certifications, Giordano serves as Exterro’s subject matter expert on the intersection of law and technology as it applies to e-discovery, information governance, compliance and risk management issues.

No Responses to ““Barney It Down”: Reviewing the Basics of Predictive Technologies within E-Discovery”

Leave a Reply