(updated Spring 2021)

The SMART Best Practices Guide for Document Relevancy and Privilege Review

  The e-discovery world is a Nine Billion Dollar industry. There are smart players and people who waste a lot of their clients' and/or Firm's money. This guide hopefully provides useful tips for efficient and cost effective ways to manage a review. Partners and associates managing review teams should read this. 

     I. Introduction

     !!. Responding to Subpoenas the Right Way

     III. Onboarding Reviewers

     IV. Know the Law

     V. Don't Pretend You Know-Ask

     VI. The Set Up

     VII. Be Available

     VIII. Color Trees and Color Highlighting Key Names and Terms

      IX. Hidden Codes

      X. Talent Retention 

      XI. Remote Work Challenges

I.   INTRODUCTION:

As a preliminary matter firms and corporations hiring directly need to understand what what e-discovery is and is not. It is not a way simply to generate a computer privilege log or put leverageable bodies on a bill. The people you are hiring are trained evidence analysists. They are not bottom feeders with no other way to make a living. They are in some some cases, the difference between you winning and losing your case or the DOJ approving your merger or not because they see and analyze first the millions of pages of documents to determine which you can withhold and which are highly relevant.  Many of them will have more experience than the associates managing them. It has become a niche specialty. 

!!.  Responding to Subpoenas the Right Way:

     This suggestion guide deals with the part of document review that is sometimes overlooked by attorneys, particularly young associates starting out on managing projects. As in anything in life there are good and bad management practices when it comes to document reviews and bad ones can tank a project quickly.  There are situations that will cost your client money they don't need to pay if you are smart at the outset. Costing money in some of these hundred person reviews can be in the tens or hundreds of thousands of dollars. 

This should be read by anyone managing a document review project new to the game.

After all the work on vender selection is done the documents are loaded and batched the hiring agency selection has happened the real management work begins. I will assume you have already verified that the hiring agency has bandwith capacity to handle your document load if your reviewers are at any agency review site or there will be serious technical issues that persistently plaque your review with downtime and buggy glitches that will require IT overtime. This guide is not a technical guide but a management one. It assumes you have full familiarity with your review tool and can demonstrate it.  The technical issues with "Remote" work present other challenges which will be discussed below in the Remote section.

 

III. ONBOARDING Reviewers.

     Increasingly this is done more and more electronically through offices of agencies in multiple cities which poses certain pitfalls. One agency let go an attorney allegedly for not having given identification on an i-9 immigration form within the first few days of starting when the office processing it was in Florida and the reviewer gave the ID copy to a DC office employee who just forgot to upload it properly to the national system. This process should be as uncomplicated as possible and there should be a real staff person either at an agency or firm available to troubleshoot glitches on electronic onboarding. Every reviewer should be given the HR email to where they lodge these sorts of onboarding documents and another one for payroll questions and issues. That would cost your client money because you had to train someone new and lost someone with valuable knowledge on that project after a few days for no reason. Those easily avoided costs just require a simple fix of providing an HR email at the outset. It can be as simple as an address like i9HR@company.com 

IV. KNOW THE LAW

     I will assume that you know the law on privilege in your jurisdiction and are familiar with the protocols on what is withholdable and not withholdable by Courts of your jurisdiction. The partner managing the review should have given you a memo on the law and that memo should be also circulated to the review attorneys on the review or part of the “privilege” section on any orientation memo. Ideally that memo is also in a drop down easily accessed on your platform, like Relativity. The privilege law can get tricky in areas and your firm may have preferences on them dictated by case law on things like should all “draft contracts” of a certain nature be marked privileged where there is no visible actual attorney on the contract or only one signature? What about this: both signatures are present but there is clear marginalia after the signature and no cover email? Each privilege question requires that there is a company wide/firm wide consistent response. Should contracts with only one signature be considered “final” and not privileged or do they need to have two signatures; one from each side of/party to the contract. There are fine tunings and high level differences of opinion of these privilege questions that usually come up in any review and its helpful to know what they are and have a memo handed out to the review attorneys on it in advance. "Common Defense" privilege participants have to be identified.

Review attorneys can generally tell which firms are experienced in reviews or not by how well they have addressed these issues in advance. They are easily anticipated and come up in virtually every review. Your firm should have a memo on privilege that addresses them. The more seasoned reviewers will know you are not on top of the game if you don't.   

 V. DON’T PRETEND YOU KNOW-ASK

     They do not teach you in law school how to manage a team of people and some people just out of law school really do not have the first clue or a project management bone in their body, yet this seems like exactly the people that get tossed onto document review management by law firms. So this is a hopefully helpful Heloise hint guide for success and “what not to do” guide more than anything. Don’t pretend you know how something works. Ask questions. Sometimes the document review team actually knows more than you do what should be in the review. Some do this for a living and its their profession. Its perhaps only your distraction from researching the next legal memo or brief possibly on your way up to full equity partner. Don’t treat it like you got the short straw.  It is highly advisable to get a seasoned document review project manager in to make sure everything is optimally set up or on consultation with vendors before you start the reviewers. This saves a lot of time and money and can reduce errors dramatically.  Its a smart idea to hire full time E-Discovery Project Managers. 

     Document review has grown into a specialized field with an entire subsection of the litigation section of the DC Bar devoted to it. Georgetown University Law Center has an annual E-Discovery Institute raising the field’s academic relevancy. In the internet paperless age where more or less all communications are electronic discovery has necessarily been computerized and becomes a critical part of evidence gathering. Your last generation partner may not even barely know how to type so it behooves you to get expert advice from document review professionals before running a review.

 

VI. THE SET UP:

   As a preliminary matter there are essential things that should happen before you call your team together or that need to get done before anyone starts a review. I really cannot stress enough how critical this step is and how insane it is not to have this at the outset of a review, while shockingly it happens often enough to warrant this warning. Review Attorneys will think you have never done this before if you don’t do these things. First it is essential that you provide the review attorneys with five documents:

 

 1. The full list of attorneys in house at your client corporation and outside counsel by firms and by attorneys. This list will be augmented as more attorneys are discovered in the documents but a basic understanding of which ones will break a privilege is essential. Who is on our team? Who is on the other side? That has to be basically grasped at the outset;

2. A list of terms or acronyms expected and often used in the trade (also will be augmented); 

3. The actual document review requests, the actual subpoena or what is detailed in any Agency inquiry letter; and

4. What classification of documents, including by numerical reference to the requests for production they reference, each tag on the review tool represents if the review involves tagging for relevancy or categories represented by review requests.

5. The Complaint, subpoena and/ or Inquiry letter if it is different than the document request document.

 

   Ideally these five documents should be loaded onto your case file section of your review tool. Relativity has a tag where these can be loaded and dropped into drop downs where people can easily access them and save them to their computers. They should also be handed out in hard copy as well.  The review attorneys starting the review should be given those five documents at the outset. They very likely will think it odd if this does not happen.

 

   Both the Attorney List and the Acronym lists should be supplemented as these names and terms are discovered through the documents. There should be a mechanism whereby they are reported to be added regularly as the review goes along to a person predesignated as that person who supplements these lists to whom updating emails can go.

  

Reviewers should be given enough time to read and digest all of the initial case documents. It is advisable that the attorneys spend as much time as they need the first day going over all the documents after hearing a presentation on each of the issue tag categories and digesting them all and that they not start actually coding until the second day. When people are rushed into starting coding they often do not fully read or digest all the materials. The first day of the review needs to be just a relaxed opportunity to receive, read and digest the materials, familiarize themselves with the review tool, and ask questions of the associate on staff assigned to the review. On one complicated multibillion dollar Antitrust review a long presentation concerning what the merger was about was presented by three firms managing the review, review attorneys then went to their review site, settled in with their binder of materials, and spent the afternoon reading, highlighting sections and asking questions which took the remainder of the day. This time should be expected.  Serve bagels and coffee. Its study time.

  

VII. BE AVAILABLE

  It is important that someone assigned to the review from the firm is available and accessible for the reviewers questions for at least a few days preferably on site at the review center.

   It is important for the Firms to understand that Agencies will often prohibit the review attorneys directly emailing the associates even when the associates invite the communication, sometimes on threat of firing. The agencies don't want their reviewers poached by firms. They have a hard time finding all the reviewers they need and don't want to lose anyone from the pool. If there is not a "team lead" assigned to the project through whom all questions or issues are addressed, there should be a "QA" section on each document coding panel to ask particular questions of the document at issue and another more generalized question email to which the reviewers can send questions with a running log of answers by the Firm. It is CRITICAL that this be attended to with timely answers by the Firm. If the Firm attorneys are not responding to these questions in real time, which are often questions that many reviewers will have as they progress through documents, then there will be inconsistent coding as people reach their own conclusions. That means that large chunks of coding will have to be redone. This can sometimes be caught on the reconciliations done at the end of the review. These inconsistencies cost attorney time and major money to correct. 

 

The types of documents that are seen should be communicated to make sure the coding is correct. Some (most if not all) agencies will want to minimize interaction of review attorneys with the staff associate from the firm and this should not be done for the first few days. Sometimes (usually) attorneys are juggling multiple cases and pop in and out at unexpected times to a review center. This should not be done the first few days of any review. There should be some dedicated firm attorney available for substantive questions for a first few days of a review sitting somewhere on site where reviewers can ask questions. If it is a huge review with hundreds of review attorneys there needs to be several dedicated firm attorneys available to ask questions. Access to the firm attorney should not be restricted to just a “team manager” or a designed review attorney who has propped themselves up as an expert running interference with the associate as the self appointed exclusive contact with the associate. Agencies should resist the ambitious self promoter who thinks they are the only person with QC (quality control)  experience in the room or qualified to be that person. That person is usually the great demoralizer and runs people off projects because they cannot stand the insult.

   FEEDBACK is important to give throughout the project and the sooner the better. If there are large categories of documents that are not responsive at all they should be identified early and to the extent possible batch coded out of the review. The reviewing attorneys will want to know how certain types of documents should be treated so its important to give them early feedback after asking them what they are seeing in their batches. Its helpful to have a running Q&A spreadsheet that is updated as people have questions with answers provided that day-not a week later into the review.

   

VIII. CODING TREES AND COLOR HIGHLIGHTING KEY NAMES AND TERMS  

You should have already block coded by color names of attorneys and firms on your tool. You should also have Block coded (typically in red) terms like “legal” “law firm” “advice” and “litigation” as well as all the attorney names you have. Its helpful to have adverse attorneys in blue or another color than the red for your legal team.   Often firms will forget that these terms can signal privilege internally in an email discussion. The worst thing you can do is partially highlight, or highlight only some names and leave out others which misleads the reviewer into thinking that it is not an attorney operating in legal capacity or not an attorney at all. Partial highlighting causes more lapses than if there were no coding because it is misleading and causes reviewers to overlook things relying on the coding. That is why it is critical to have a running ongoing log of attorney names that get added daily onto the highlighting and attorney list.  

A second level review in putting together the privilege log should have easy drop downs that reflect segments of the log that fit together to create full sentences. If the ultimate goal is creation of the privilege log for filing don’t assume the first level reviewer doesn’t have capacity to properly enter reasons for the privilege. Having the reasons for the privilege on specific drop down coding phrases saves time and your client money.

 

   If you are managing a review with issue tagging there are critical do’s and don’ts. The issue tags should correspond to the numbered document requests. In the event there has been anysort of negotiated narrowing of those requests or in the event that you wish to combine several into one tag or if the tags in any manner do not correspond to the actual numbered requests, then the tags need to be identified on the review tool with descriptors. Any tag panel of more than ten tags needs to have descriptors next to the tag in descriptions that are more than one word. It needs to be fully explained orally and in writing what is meant by each tag by number clearly spelled out. Ideally every tag should have this but if there are more than ten tags it is absolutely critical that they contain full descriptors for the simple reason that people cannot retain in their heads all classifications of documents that pertain to each tag while going at recommended speed through the review tool while tagging. This will dramatically improve accuracy.  Better accuracy with better speed means you are saving your client money.  If you have more than twenty tags it is advisable that you separate out that into separate level reviews by putting an “other” tag in that can go through another round of reviews to break down the “other” classifications.

 

    On one review managed by people who clearly were new to the game and not at all familiar with managing reviews they put 200 issue tags without one word of identification next to the tag identified only by number. I cannot stress enough how insane that is. It is a waste of the clients money to have review attorneys have to go back to the document requests (which in that case were in three different requests and supplemental requests in three different binders because they didn't realize they could have uploaded to the review tool while they blocked computers Word) for each document to determine which number corresponded to which of the 200 issues identified in the review panel without one word of description. Several seasoned reviewers just walked off that project.

 

IX. HIDDEN CODES

  If there are hidden codes or data there needs to be a way to access the full information. If the emails contain only names and for some reason the email address has been blocked out, there needs to be a metadata drop down so the reviewer can see the full email address. If this is not done and the reviewer is left to just guess who the person is or where they come from (which firm/company) they cannot be expected to get the privilege coding right. The metadata has to be accessible at all levels of review.

If hidden codes are revealed by another panel it needs to clearly state that on the review tool. 

X. TALENT RETENTION

   It does not serve a review or your client's bottom line to have people leave before the project ends but it happens. Retraining is an unnecessary cost. There are obvious reasons it happens that can be minimized. For starters the firms that offer the best benefits on top of the usual hourly rate will have the best success retaining people whether they work in a review center or at the firm. These reviewers have had their wages stagnant for over a decade at a small hourly wage (in DC around $30/hr) which barely meet cost of living expenses given city real estate costs. A one bedroom apartment in DC and NYC runs typically above $2,000 a month while these document reviews pay little more than $30.hr. So things that help keep people interested in working on a review include things like consideration of meals, gym memberships, commuting cost reimbursements and free parking.

     The agencies who provide bagels weekly, and regular pizza Fridays are talked about as the best agencies among reviewers. There is a reason a Passover meal signals deliverance and the last gathering with disciples was at a “Last Supper.”

    Putting a team of professionals together there are bound to be personality issues. The agency or firm in the onboarding probably already gave them a handbook on sexual harassment and discrimination policies. Its wise to provide a designated person to whom complaints can come who is not the exclusive person staffing the project for obvious retaliatory reasons. There should also be a statement at the outset that people, as they are all professionals with their political opinions, unless they are friends prior should not discuss politics on the job. This is very important in DC in particular where opinions run hot and reviewers can sometimes be volunteering for campaigns on any off hours.

Lastly, understand that people working on document reviews are not second class attorneys or attorneys who could not do anything else. Attorneys do document review for a number of reasons. It provides schedule flexibility for parents, it allows people to make enough money in a short amount of time to have months off for travel or other professional pursuits and it does not require you take any litigation stress home. Document review attorneys often have worked in large prestigious firms already, have been associates already, some have started their own firms and some are in between client settlements. They should be treated with all the professional respect you treat anyone in your firm. Underpaid does not mean substandard talent. As stated above, these are evidence analysists, some of them have Ivy degrees and top 20 law school credentials, and they are trained to know the finer aspects of privilege determinations and what is likely to get challenged on a priv log. Treat them like the professionals you are. 

 

XI. REMOTE WORK CHALLENGES

     There are advantages and disadvantages of e-discovery done "remotely" as offices shut down globally and people used to commuting were shut in home. Some used their home equipment to do these reviews and other firms provided equipment which usually came with spyware-not to spy on the computer but the reviewer.  First it should be noted that to the extent that there are reviews from a national firm ongoing in several States/jurisdictions there has always been an aspect that was "remote" as multiple cities were network linked.  Post Pandemic and the general virtual shut down of offices and Agencies in major cities, "remote" took on a new meaning. Reviewers were driven to their home offices or basements or the nanny's room to set up the requisite equipment. Reviews are ideally done on two screens and the best agencies have typically provided two screens so multiple tabs can be opened for review, coding, reviewing the protocols and names directory. That means that the reviewer home office has to have capacity to set up two screens on an efficient computer system free from distractions like rambling toddlers, attention seeking canines and other family distractions. They have to purchase that equipment. No one has an excuse for being late for dinner. No one can say "I am working late" as an excuse not to come home.

    To be sure, avoiding commuting, parking or metro fees and time is advantageous -more hours can theoretically be billed. So what are the downsides. 

    First, reviewers might not want the spyware that comes with borrowed laptops or having to download something that intrudes in their home spying on their productivity. Second, if they use their own equipment, they don't get the IT repair help support. Reviews that go on for weeks or months are wear and tear on equipment that may be their only computer. They may have to go to the Apple Store for repairs or repurchase parts (cables, monitors, etc.) at Staples, at their expense. When the Agencies or Firms provide the equipment, or lease it, they control the IT, they control the shelf life of the equipment and they can service them on the spot with the best IT professionals. An individual cannot do that. They have to waste time with the apple geek desk or Best Buy computer help. They have to pay for it. Individuals are eating what should be the Agencies or Firm's overhead. There is a basic unfairness feature to making people who make $30/hr do that. 

     What are the practical downsides from a substantive perspective with home reviews? The biggest pitfall is that there is not real time simulteneous feedback when there is no associate in the room and no collaboration between reviewers. As stated above the largest driver of inconsistent coding is lack of timely associate/firm feedback on documents over which many people have questions resulting in different outcomes. Inconsistent coding will cost the client more money than they bargained for and it can even threaten the Firm's relationship with the client when the project comes in three times more expensive than projections. Firms try to mitigate this with elaborate drop down question and answer panels or logs but this cannot replace on site collaboration. 

   So on balance, Remote work is not preferred to in office e-discovery reviews where people are staffed together discussing documents and issues as they arise. If Firms hope to keep their clients, they need to control better the costs by controlling the inconsistencies inherent in multiple people doing things from different locations. Attorneys on reviews,as they see documents, discuss amongst themselves various intracacies and calls regarding the documents before them to get on the same page. When that cross collaboration is missing with the missing in action associate who will get back to the Q&A log maybe the next day that creates massive amounts of inconsistent coding that has to be redone raising the costs, sometimes astronomically. 

   There are also differences in wifi support, some of which isn't much support. Some people are on plans where there is "buffering" that slows down a project as they have to log in and out multiple times. There is a difference between a basic Verizon wifi and the Optimun enhanced that is available in other regions. Download and uploading speeds affect the time in which documents can be done. Again-that is money to your client and you.

If you want to keep your clients and as a professional obligation to keep costs from duplicating needlessly, firms need to get back to offices where reviewers work collaboratively communicating with each other, the IT support is johnny on the spot with the best optimized broadband with capacity for complex platform loading, and associates regularly interact with the reviewers. The reviewers should not have to cover the Agency overhead in the form of equipment and IT support. 

In short- Remote was nice while it lasted but it shouldn't. Not in the legal e-discovery review context. Its penny wise pound foolish.  

The Mayor of New York City and Governor both are urging businesses to get back to work downtown. The pandemic levels are now roughly comparable to the other leading causes of death in the State: Cancer and Heart Disease. Agencies and Firms can easily install plexiglass dividers between reviewers, mandate vaccines to come to work downtown and have unvaccinated people off site. They can and should immediately gear up for proper in office reviews. There really is no excuse not to.

 

 

Best of Luck with your review. 

     

 

 

These helpful Heloise suggestions are by: Cynthia Butler, Esq. Cornell University and Georgetown University Law Center graduate, member of Women in E-Discovery who has worked in the field off and on for over a decade with top DC and NYC law firms on, inter alia,  major mergers prosecuted by the US Department of Justice with special thanks to the E-Discovery teams at Sidley, Crowell and Arnold and Porter, Kaye Scholer.