The Heart of the Matter: Cutting E-Discovery Review Costs by Tracking Relevancy Rates (and Other Metrics)

Created on June 2, 2017

Former Content Marketing Manager at Exterro

Exterro Comprehensive Guide to E-Discovery Review

With different fact patterns, laws, custodians, and companies, every review is going to be unique. When outside review teams are used for a company, multiple groups are doing different things in different ways, and it's very difficult for corporate counsel know exactly what is working.

A repeatable process allows in-house legal and the outside law firms and/or vendors doing the review to compare information or feedback, ensuring the result will be consistent. This requires planning and defining roles and responsibilities for all involved.

Create Communication and Transparency Between All Stakeholders

For checks and balances to work, multiple teams must have transparency and great communication within the preservation and collection process, which may include In-house legal, technology vendors, outside counsel, and the review team. Without communication, transparency, and a knowledge of scope, inefficiencies begin to show up in the process.

Some legal teams utilize an e-discovery lead or an e-discovery specialist that acts as a conduit between IT and legal. Legal knows what they want to preserve, at least at a fairly high level, and IT knows where all the data lives. Between the two of them, they must make sure they're on the same page and understand exactly what they're talking about. This is where a liaison, who can speak both legal and tech, can avoid a lot of problems.

Another way open communication can create a more repeatable process, thereby saving on review, is instead of thinking of each matter as its own unique instance with its own preservation and/or collection set, teams should strive for the kind of transparency that would give them knowledge of what has already been preserved for other matters, whether they are similar in nature or not. If there is custodian overlap between matters, some of the same data may already have been locked down, so there is no need to duplicate that effort.

Tracking Review Relevancy Rates

Even after narrowing the funnel of data in the preservation and collection stages, there is still irrelevant ESI that gets through to the review stage. Review Relevancy Rates measure how much of the data sent for review is relevant to the case. If the relevancy rate is 1%, it might be because this was a government investigation where a very broad preservation and collection couldn’t be avoided, or it may have been something where targeted preservation or targeted collection wasn’t used. Once the things contributing to that 1% relevancy rate are located, checks and balances can be put in place upstream in the hopes of improving those rates, which in turn will cut costs.

If review isn’t being done in-house, it’s important to ask the outside review team for these metrics. This goes back to creating transparent lines of communication. This knowledge should be transferred back and forth, so that reviews going forward can be more effective and efficient.

Using Metrics to Track Review Performance

A lot of flexibility and granular information is needed in order to evaluate performance, but more important is the ability to quickly collapse that information into a meaningful report. This goes back to the idea that every case and every review is unique, even when working with the same enterprise client or law firm. There are different angles and things that an attorney, or in-house counsel, or a service provider is going to care about as they're looking at this data to understand performance.

Another area where metrics can help are with budgets, which are surprisingly often overlooked, even with the knowledge that review is the most expensive e-discovery stage. With the metrics that review tools can provide, the project can be looked at when it comes in—data set, time it will take to review, the exact numbers in terms of relevant and privileged ESI. But what is forgotten in many instances is continuing to track budgets during the review. When budget concerns are addressed as the project goes along, then calls from in-house counsel saying, "What happened, this is three times what you projected in your budget," are avoided.

By keeping everything transparent with all stakeholders, then even if a budget projection based on a document set changes after getting into the data and conducting ECA, everyone can be kept up to date in real time. It's a collaborative approach. In-house counsel will be interested in changing budgetary issues. Outside counsel will be interested in how changes in the number of reviewers or size of the dataset will affect deadlines. The same numbers may be reported to both, but they mean different things to different players.

Getting More from your E-Discovery Software During Review

  • Custodian Interviews and In-Place Preservation

Just before issuing legal hold notices, and after as well, custodian interviews are vital for understanding what data they really have, and how much they are aware of the claims of the case. Not only can technology make the interview process easier and more transparent, it can also alleviate the concern about the growing number of data types, as well as data volume, with tools like in-place preservation (Office 365 is a great example of IPP capability, where all data stored in Exchange, One Drive, SharePoint, etc, is automatically preserved without making duplicate copies).

  • Sampling / Targeted Collections

Take advantage of Early Case Assessment tools, while using more advanced search techniques, automation for privilege calls, or leveraging machine-learning to help identify relevant documents. Other powerful tools can use analytics for things like concept clustering and email thread analysis.

  • Using a Unified Technology Platform

Using multiple e-discovery tools creates a lot of inefficiencies. The ideal is a single end-to-end platform that orchestrates all elements of the e-discovery process, putting the tasks together from preservation through review, including customizable reporting features for all those tasks, which can continue throughout the life of the project. Combine all this with managing and optimizing the processes for the human aspect review, and the result is full orchestration over the entire e-discovery process.


Much can be done before, during, and after the review stage to gain performance and cut costs, and much of that has to do with mending fractured processes and lines of communication between the stake holders so that knowledge can be more effectively shared, resulting in smaller datasets, fewer fire drills, smoother handoffs, and a more efficient approach overall.

For more on how you can create a more efficient and cost-effective review process, Download Exterro's Latest E-Book, The Comprehensive Guide to E-Discovery Review