eDiscovery AI Relevance Review

This article guides you through the process of conducting a relevance review using our AI-powered predictive coding tool, integrated seamlessly with Relativity.

1)   Prerequisites

a)     System Requirements

i)       Before beginning the relevance review process, ensure you have the following:

(1)   Extracted Text: All text based (i.e. emails, word docs) documents must have extracted populated in the Extracted Text field.

(2)   Native Files – For image or Audio review the documents must include native files. eDiscovery AI currently supports the following native file types: bmp, fpx, gif, ico, jpg, jpeg, png, pntg, ppm, tga, tiff, wav, mp3, mov, ogg.

ii)     File Size Limitations: All text-based documents should have an Extracted Text Size of less than 350 kb. Audio files may not be longer than 12 in length.

iii)   Relevance Criteria: Clearly defined criteria for what constitutes a relevant document for each issue in your case.

iv)   Single Choice Fields: A designated single choice field in Relativity for storing relevance determinations. You need to create the field and choose it from the dropdown, but eDiscovery AI will create the choices.

b)   Permissions

View necessary Relativity permissions here

2)   Components

There are several key components that are part of the eDiscovery AI tool:

a)     Job Name

i)       A user created name to help track and identify each submission to eDiscovery AI for tracking and reporting purposes. This field is optional, but is best practice to include a unique name for every submission.

 

 

 

 

b) Prompts

Custom instructions given to eDiscovery AI to define Relevance for each issue. The prompts for each issue must be no greater than 2,000 characters. The language in the prompt does not need to match the language of the document, i.e. English prompts will work on Korean documents. Note, by default there are 5 available issues that can be run in one pass.  This can be expanded up to 15, view information about adding additional issues here.

 

c) Fields

i) The Relativity field where eDiscovery AI's relevance determinations will be stored. This is a single choice field that must be created by the user, but eDiscovery AI will create the choices.

d) Summary

i) eDiscovery AI will create a concise summary of each document. This summary is typically 2-3 sentences and is not specific to any individual issue.

a)     Relevance Coding

i)       eDiscovery AI’s determination of whether a document is relevant or not, based on the criteria from the user’s prompt. There are 4 potential choices: Relevant, Not Relevant, Needs Further Review, or Technical Issue.

(1)   Relevant – This document meets the criteria for relevance as defined in the user’s prompt

(2)   Not Relevant - This document does not meet the criteria for relevance as defined in the user's prompt

(3)   Needs Further Review – eDiscovery AI is unable to determine if the document is relevant or not based on the prompt and the contents of the document. These documents will require manual review.

(4)   Technical Issue – For some technical reason this document was unable to be reviewed for relevance for at least one of the issues.

(a)   Possible reasons for Technical Issue coding

(b)   The document exceeds the maximum extracted text size, or file size

(c)   The document is not one of the supported file types

(d)   The document is missing the necessary file format

(e)   Audio or image file missing Native

(f)    Text based document missing extracted text

b)    Explanation

i)       A short, 1-2 sentence explanation from eDiscovery AI describing why it made its relevance determination. A separate explanation will be created for each issue and the explanations will be stored for each time a document is run through eDiscovery for an issue.

ii)     The explanations are recorded in a custom object within Relativity. By default, a document layout called eDiscovery AI layout will be created that will contain all the explanations for all issues and runs of that document. This can also be added to any existing document layout. 

 

 

3. Process

Follow these best practices for an eDiscovery AI Relevance Review:

a) Select Documents for Review

Select a set of documents and use the Mass Action Send to eDiscovery AI to launch the pop-up window.

 

b) Draft Prompt

Create an initial prompt that instructs the AI on how to determine relevance based on the criteria for each issue. Type or paste each prompt into the issue box and select the corresponding field for the relevance determination for that issue. Each prompt is limited to 2,000 characters. For more information related to prompts see Prompt Best Practices.

a)     Run Initial Prompt on Document Sample

i)       Run a small sample of documents through eDiscovery AI using your prompt. The sample should typically be at least 200 documents and contain plenty of both positive and negative examples from which to draw feedback.

ii)    Review Sample Results

(1)   Carefully review all of the sample documents and compare the manual review coding to eDiscovery AI's coding. Take note of any documents that were coded incorrectly and the likely cause of that incorrect coding. The coding explanations are particularly useful in this process.

iii) Refine Prompt

(1)   Based on the manual sample review of the documents that were coded incorrectly, adjust the language of the prompt to improve accuracy and address any issues identified.

b)    Re-run Against Sample

i)       Apply the refined prompt to the same sample to verify improvements. The purpose of re-using the same sample documents is to limit the amount of human review necessary, but it would also work to run a new sample of documents as well.

c)     Final Run Across All Data

i)       Once satisfied with the sample results, run the AI analysis across your entire document set. If the entire document population is larger than 100,000 documents it is typically faster to submit the documents in multiple smaller sets rather than one large group.

2)  Results

After the documents have been submitted to eDiscovery AI you can monitor the progress and work with the results:

a)     Tracking Progress

(1)   Monitor the progress of the AI analysis in real-time through the eDiscovery AI Request tab in Relativity. Learn more about using the eDiscovery AI Request tab here.

b)    Using eDiscovery AI output in Relativity

i)       eDiscovery AI fields and objects can be added to your Relativity Layouts and Views like any other object in Relativity. 

ii)    Add Fields to a Layout

(1)   To add any of the eDiscovery AI fields to your layout, scroll or filter until you’ve found the field(s) you want to add, then drag them on to your layout. 

 

 

a)     Working with Default Layouts and Views

i)       A layout will automatically be created in Relativity called eDiscovery AI Layout that includes the explanations for each issue, the name of the field used for that issue as well as the prompt the user entered for that issue. Unless the application is unlocked, even users with appropriate permissions may not be able to edit this view.

ii)     To unlock the application, navigate to the Relativity Applications tab. Click on eDiscovery AI Relativity and in the right-hand corner click on the Unlock Application button.