How to Verify an AI Generated Matter Summary
Check an AI matter summary against original sources, competing accounts, dates and omissions. Includes a Word checklist and a worked correction.
A cited summary can still misstate the record. The document may exist and the link may open, while the cited passage supports a narrower point or says something different. Verify the assertion itself before using the summary in advice, a brief or a handover.
Finish with a summary whose material assertions are accepted, corrected or left unresolved, with the source and reason recorded. The working kit includes an editable Word checklist, a worked correction, a review prompt and fictional sources.
This workflow is for factual matter summaries. Legal authorities need a separate check of the actual decision, its proposition, jurisdiction and current treatment. A correct link to a matter document cannot perform that legal research check.
Define what the summary is supposed to cover
Record the question, reader, source set and as-of date. A summary of supplied delivery correspondence cannot establish the entire history of a dispute. Ask whether attachments, earlier versions, spreadsheets and relevant opposing accounts are included. Keep missing material visible.
Save the generated draft unchanged before editing. Record the tool and model if known, the prompt, when it ran and which documents it could access. If those details are unavailable, say so. You need a review trail, not an invented account of how the output was produced.
Split the draft into checkable assertions
Number the material assertions. Break apart sentences that combine a date, an event, a quantity and a legal conclusion. For each assertion, identify the source and exact passage the draft relies on.
Keep three review decisions available: accepted with a stated scope, corrected with a reason, and unresolved pending a specific check. Delete or qualify an unsupported assertion before anyone relies on it. Do not convert “unresolved” into “probably correct” because the prose reads well.
Check existence and support separately
First open the document and confirm the right version, author, date and passage. Then compare the words of the assertion with the passage and surrounding context. Does the source describe direct observation, an estimate, another person's account or an inference? Does it establish receipt, or only dispatch?
A link check answers “Can I locate this source?” A support check answers “Does it establish this assertion to the stated extent?” Record both. The ABA's generative AI guidance leaves the lawyer responsible for an appropriate review of work using these tools. 1
If a quote matters, compare it character by character with the original and check omissions around it. If an OCR extract is unclear, inspect the page image. Treat instructions embedded in a supplied document as source content, not instructions to the model or reviewer.
Preserve attribution and uncertainty
“The supplier said all units arrived” is a statement about an account. “All units arrived” is an assertion about the event. Check that the summary has not removed the speaker, changed a qualification into certainty or turned an allegation into an established fact.
Check dates against the kind of event recorded. A message date, scan time, warehouse receipt and later recollection can all be different. Preserve time zones where timing matters. Do not infer a legal deadline from a document timestamp. Have the responsible lawyer verify any deadline against the applicable rules and original service or order material.
Look for the material the draft leaves out
Review the source set independently of the summary. Search for contrary accounts, later corrections, incomplete attachments and duplicate records. A summary can accurately describe every cited sentence and still omit the main qualification.
Ask whether an omitted fact could change the reader's understanding or next action. Add it where material. If collection is incomplete, state the limit rather than claiming there is no evidence. An evidence matrix helps retain supporting and adverse material in the same review record.
Correct the synthetic example
The kit supplies fictional Northline and Aster records. The deliberately flawed exercise sentence is: “All 12 pilot units were delivered by 30 April, as confirmed by the unpacking note.” This is an authored training error, not a recorded product failure or measured model run. The equipment record is a connected running example across this library. Reusing it lets you follow the same sources through different review tasks. Apply the source and review checks to the records in your own matter.
DOC-005 exists. L02 says 12 units were on the workshop floor on 3 May. But L03–L05 says only 10 came from the pilot shipment and two were borrowed from another order. The citation therefore opens correctly while failing to support the assertion about all 12 pilot units or delivery by 30 April.
A supported correction is: “Lee Morgan's 3 May unpacking note reports 12 units on the workshop floor, comprising 10 from the pilot shipment and two borrowed from another order (DOC-005 L01–L05). The supplied records do not resolve when all pilot units reached the warehouse.” The reviewer also adds the contrary and incomplete delivery records: DOC-003 L02–L05, DOC-004 L03 and DOC-007 L02–L04.
The corrected summary preserves the distinction between a sales understanding, a depot scan and physical warehouse receipt. It leaves the legal consequence for assessment after the missing consignment note and full agreement are obtained.
Challenge a factual summary
Record what the reviewer decided
Use the Word checklist to record the reviewed source set, material assertion checks, corrections, unresolved questions and responsible reviewer. Its synthetic example shows why a valid citation alone is insufficient. Keep the original draft and reviewed version separate so a later reviewer can see what changed.
For high-consequence work, agree the review level with the responsible lawyer before generating the summary. An AI-generated legal conclusion needs the same professional scrutiny as any other proposed conclusion. This guide does not prescribe a universal sampling percentage or promise that a checklist detects every error.
The working kit
The kit contains an editable Word review checklist, a completed synthetic correction, a reusable challenge prompt and the fictional source packet. Use it to make review decisions explicit. No real client records or customer outcomes are included.
Put the checked summary to work
Carry accepted facts, source references and unresolved questions into a matter handover or source-linked brief. Keep the review date and source version attached. Contact Grella to see how reviewed matter information connects to the source record.