The ability to measure results is one of the major benefits of digital legal transformation. Generally speaking, any legal work done using a tool generates data. Computers are exceptionally good at the mundane accounting for time spent without it taking extra time. As I’ve pointed out before, this type of metadata (data about the legal department’s performance) is the key ingredient that makes digital transformation more than just a time-saving measure. Instead, by leveraging the extra information generated, the solutions used by legal teams become means of identifying potential improvement.
With new tools claiming their place in the legal process, Legal teams are better equipped than ever to measure and improve their performance. This isn’t all just extra work, either. By having data available, budgetary changes can be requested or more easily justified. Businesses as a whole have increasingly been making data-driven decisions for a long time. The fact that legal has not joined in on this, is mostly due to the unusual position it occupies: a protector of interests rather than a revenue generator. Expected cost is one thing, but it’s not like a chief legal officer can join a budget meeting in January to say “we expect to be involved in a massive litigation battle with a competitor by Q3 of this year.”
Contents:
Why measurements matter
1. Ask the IT guys for help
2. Start at the beginning
3. Dissect successes and failures
4. Reassess your process
Download the whitepaper to learn how eDiscovery helps you easily sift through huge volumes of data and find all relevant documents for a case in a fast and cost-effective manner.
Why measurements matter
The above doesn’t mean legal budgets are untouchable. In previous blogs (this one, or this one) the bind in which corporate legal finds itself has been discussed. In short: deadlines are tighter, while data volumes continue to grow. In essence, more to do in less time, and with human productivity topping out, two options remain:
Increase the headcount: growing the team to handle a larger, more time-sensitive workload makes sense at first glance. After all, more legal professionals can spend more hours doing more work. However, it can be a very hard sell: larger teams are not only more expensive, they are also more error-prone. Coordinating a small army of lawyers collaborating on a single legal matter inevitably creates more admin. It may even require an extra layer of quality control on top of the initial review to weed out inconsistencies between team members. Of course, that extra level of control will eat up most of the time saved by growing the team. Failure is by no means guaranteed, but the risk of a self-reinforcing downward spiral is very real.
Digital Evolution. The other option is the famous maxim from motivational posters everywhere: work smarter, not harder. By using the available technology to take over some of the more tedious or repetitive tasks, the legal team can simply do more in the same amount of time. This also applies to workflow management.
Doing this step-by-step helps avoid disruption. This is possible without increasing the already topped-out productivity of legal team members. The data generated by the tools you use can be used to evaluate performance, identify areas of improvement, and budget smarter.
The goal for the now data-generating legal department is to figure out how to use those metrics to maximize performance (the bang) in the context of the budget (the buck).
Here are four things to definitely keep in mind:
1. Ask the IT guys for help
Formulating a KPI is one thing, being able to measure it is another. Many tools generate a sizable amount of ready-to-use data about their own performance. Process-wide information may be slightly more difficult to obtain. When a KPI is formulated that doesn’t have a ready-made measurement, legal should move to involve IT to see what the possibilities are in terms of collecting cross-platform information and procedural metadata. Generally speaking, IT possesses a lot more experience and know-how when it comes to gathering data: there’s no need for legal to reinvent the wheel. The information gathered will typically not be sensitive (in a legal case anyway), since it’s about the performance of the legal department, not about the specific case.
2. Start at the beginning
When it comes to improving a multi-phased process, it makes the most sense to start at the beginning. This seems obvious, but the beginning may not seem like the most pressing issue when the initial data is being gathered. The information you gather at first is likely fairly basic, mostly useful to establish baselines and issues. For example:
- The cost of collecting electronically stored information (ESI);
- The cost of processing and hosting data from its native format into a format that can be loaded into a document review system;
- The cost of reviewing the data for responsiveness and privilege;
- The cost of producing responsive, non-privileged ESI.
Having these in place will help to identify what area to focus on for the most significant improvements. In most cases, Document Review will be the major sticking point. Although the numbers vary wildly, eDiscovery accounts for up to 50% of litigation costs. As a part of the eDiscovery process, document review eats up about 75% of total eDiscovery costs. Starting with document review might be tempting, but as a general rule, it’s best not to if issues exist in stages prior. As one of the middle phases, document review relies heavily on the collection process, for example.
Early inefficiency tends to have a disproportionate impact down the road. For example, Legal Hold is an area many corporate legal departments struggle with. Aside from spoliation concerns, ineffective Legal Hold can slow the entire eDiscovery process to a crawl. If the collection stage is indeed an issue, then it’s worth focusing on that first. No need to worry about review or production when collection is an issue.
When evaluating the legal data collection process, here’s a few data points worth looking at:
- The average number of custodians by type of case. For instance, key custodians versus secondary or tertiary custodians;
- The average volume of data potentially requiring collection per custodian;
Variances whether corporate versus field custodians are involved
Variances based on the type of data required for collection. Email versus non-email versus structured data. - The average filtering percentages and the levers affecting filtering percentages;
- The average turnaround time and cost-per-custodian all-in and broken down by:
Preservation
Collection
Processing and hosting
Review
Production - The average recall and precision numbers broken down based on applied filtering percentages
Once you’re confident the collection is sound, move on to review. If review is sound, move on to production. There’s no point in fixing the engine if you’re filling the tank with the wrong type of gasoline.
3. Dissect successes and failures
When evaluating results, the temptation to focus on the failings in the process is great. Reexamining the steps taken and identifying where the process isn’t working as smoothly as it can is useful, but not the only way to improve. Taking notes on the successful parts of the process can be incredibly useful, for a number of reasons:
- Contributing factors to success in one phase may be used in other phases that don’t run as smoothly;
- Net positive actors. Efficient reviewers, responsive custodians, can be used to train others. Debriefing such actors on their process may yield useful learnings;
- Even well-functioning parts of the process may still be improved. Gathering metrics and formulating an improvement plan is worth doing.
This also applies to cases. Obviously, a legal case that is plagued by issues and comes in way over expected budget will be assessed. But don’t forget about the ones that are run smoothly and meet all the budgetary expectations. Understanding success is just as important as remediating failure.
4. Reassess your process
Finally, it cannot be said enough that making use of metrics and analytics to improve is not a fire-and-forget endeavor. The commitment to improve is an ongoing one, and the work is never done. Every case you work on provides new data and insights to your framework. Each one can open new avenues for further improvement. Over time, there are new data formats to contend with, new custodians and reviewers to manage, etc., etc.
The initial flurry of improvements will wane, of course. Legal professionals can be conservative when it comes to changing their processes. As I noted in my earlier blog on digital transformation. By nature, the legal department is risk-averse and will avoid disrupting the apple cart. Doing what works, and dealing with expected negative consequences, is preferable rather than trying something new and dealing with unexpected ones.
However, by methodically reassessing the eDiscovery process and addressing small inefficiencies, legal can avoid larger disruptions in the future. If history teaches us anything, it is that change is inevitable: it is only a matter of how much change there is to manage. Continuously assessing and reassessing the process lets you take measured steps, and saves you the trouble of a leap of faith down the road.
Over to you
Ready to establish eDiscovery metrics and start learning from your data? Contact us today, we'd love to hear from you.
