How to Run a One Matter Legal AI Pilot
Test one bounded legal workflow against a baseline, inspect source support and measure total review effort. Includes an Excel scorecard and decision log.
Choose one matter task, define what a good result looks like and measure the review needed to reach it. A one-matter pilot can show whether a workflow deserves a wider test. It cannot establish performance across every matter, prove a security claim or justify a firm-wide rollout by itself.
Finish with an evaluation record showing the task, source checks, measured effort and reasons to stop, retest or expand the trial. The working kit includes an Excel scorecard, quality checks, calculation examples, a test-design prompt and fictional sources.
The scorecard helps compare a manual baseline with an AI-assisted attempt. It leaves actual measurements blank until you run the work. Its completed example uses explicitly invented numbers to demonstrate the formulas, not Grella results or a model benchmark.
Choose a bounded task and approved material
Pick a task with an inspectable output: a chronology of a defined period, propositions for a defined issue or a factual summary of a stated source set. Avoid “understand the whole case” as the first acceptance criterion.
Use material your organisation has approved for the specific tool and purpose. If privileged or confidential material is not approved, use a synthetic packet. Do not infer permission from a public product page, a colleague's account or this guide. Grella's public content does not itself establish approval to use privileged material.
Record the source set, collection gaps, expected output and excluded work. Name the responsible reviewer and decision owner. Security, privacy, retention, training use and professional obligations need their own verified review before sensitive data is used.
Set acceptance criteria before the run
Create test questions the source set can answer. Include a straightforward extraction, a competing account, a date or quantity ambiguity and a missing-record question. State what an acceptable answer must preserve and what it must not claim.
For the synthetic equipment packet, a useful test asks whether all 12 pilot units reached the warehouse by 30 April. An acceptable answer distinguishes the sales account, carrier depot scan, carton receipt and unpacked unit provenance, then leaves complete warehouse delivery unresolved. The equipment record is a connected running example across this library. Reusing it lets you follow the same sources through different review tasks. Apply the source and review checks to the records in your own matter.
Agree how you will treat critical errors. For a relied-on summary, an invented source, an unsupported material conclusion or a confidentiality breach may be a stop condition. Define those conditions with the responsible lawyer. Do not hide them inside an average quality score.
Measure a manual baseline on the same task
Record the time to locate sources, prepare the output, inspect material assertions and correct the draft. Describe the reviewer's experience and how the task was performed. Use the same output scope and review standard for the assisted run.
Avoid comparing a reviewed manual product with an unreviewed AI draft. If the same person performs both attempts, they may learn the packet during the first run. Record the order and that limitation. A wider test can vary reviewers, tasks and order to examine whether a result holds.
Record the assisted run accurately
Save the prompt, tool, model or version where available, date, source inventory and raw output. Time setup, generation, source inspection and correction separately. Include reruns and failed attempts where they are part of the workflow being evaluated.
Do not replace missing measurements with zero. Leave them blank and explain what was not recorded. A zero-minute review is a measurement claim, not a placeholder for unknown effort.
Check quality before calculating time savings
Inspect each material assertion against the original record. Record source existence, actual support, correct attribution, date and quantity handling, adverse material, omissions and unresolved questions. Keep legal authority verification separate from factual source checks.
The NIST Generative AI Profile provides a primary framework for examining risks such as confabulation and evaluating a system's limits. Use a task-specific evaluation rather than treating a fluent answer as proof of reliability. 1
If an assisted output fails the agreed quality gate, record that failure even if it was fast. A corrected output can still be useful, but count the correction effort. Do not erase the first failure from the run history.
Read the synthetic scorecard correctly
The worked scorecard is an arithmetic illustration only. It invents a 60-minute manual baseline and an assisted run with 10 minutes of setup, 5 minutes of generation, 30 minutes of source review and 10 minutes of correction. Total assisted effort is 55 minutes, so the illustrative difference is 5 minutes, or about 8.3% of the baseline.
Those numbers were not observed in Grella or any external model. They show why generation time alone is the wrong comparison. The workbook also includes a failed run with a critical unsupported assertion; its decision is “stop and inspect” regardless of the displayed time difference.
The blank scorecard is where your actual results belong. It calculates total effort and differences only when all required timings are present. Missing quality decisions leave the result pending.
Define checks for a bounded pilot
Make a decision with its limits attached
Choose stop, revise and retest, or expand the evaluation. Record why, what failed, what passed and which evidence is still missing. A one-matter pass is a reason to test more representative work under approved conditions, not a general reliability or compliance claim.
Keep the evaluation packet: sources, prompt, raw output, reviewer corrections, timings, quality checks and decision. If a source changed during the test, record it and repeat affected checks. Have the decision owner review the complete record.
The working kit
The Excel scorecard includes a blank run log, quality-check register, acceptance criteria, synthetic calculation examples and field instructions. It includes the test-design prompt and fictional source packet. Formulas remain pending until required inputs are supplied.
Start with a workflow you can inspect
Use the evidence matrix or summary verification guide for the first bounded task. Contact Grella to discuss a matter workflow, without treating this guide as a performance promise.