11/08/2026

Audit Trails and Citations: Making AI-Assisted Archival Research Verifiable

There is a moment in every AI-assisted research project when the tool produces something wonderful — a connection between two documents nobody had linked before, a name resolved across three spellings, a date recovered from a damaged page. And then comes the question that decides whether the project is scholarship or a party trick: how do we know?

This post is about the unglamorous machinery behind that question — provenance, audit trails, and citation discipline. In our experience it is the single biggest difference between AI projects that archives and researchers come to trust and ones they quietly abandon.

An answer without a source is an opinion

Large language models are fluent enough that their unsupported statements feel like findings. They are not. For archival and historical work we treat every model output as carrying a burden of proof, and the pipeline — not the researcher's memory — has to carry it.

Concretely, that means every claim that reaches a report, a catalog record, or an annotation must be traceable to:

  1. A document — the archival item, with its identifier or shelfmark.
  2. A location within the document — page, column, line region, or timestamp for audio.
  3. A processing history — which scan, which OCR pass and model version produced the machine-readable text, and with what confidence.
  4. A verification status — has a human looked at this exact passage, or is it still machine-only?

If any of the four is missing, the claim is a lead to investigate, not a result to publish.

Why "the model said so" fails in this domain

The failure modes are well known by now, but they bite harder in historical research than almost anywhere else:

Hallucinated citations look exactly like real ones. A model asked for a source will happily produce a plausible archive name, a plausible fond and file number, and a plausible page. Nothing about the form of a fabricated citation distinguishes it from a genuine one. The only defense is structural: citations must be attached mechanically by the retrieval layer, never generated by the language model itself.

OCR errors propagate silently. If the text layer misread a name or a date, every downstream inference inherits the error with full confidence. This is why we treat confidence calibration as a first-class output of the OCR stage, not a debugging detail — the calibrated score travels with the text through the whole pipeline.

Historical claims are asymmetric. A wrong price in a product catalog gets corrected by an annoyed customer. A wrong deportation date in a testimony summary can stand for years, be cited, and cause real harm. The cost of an unverifiable claim is not evenly distributed across domains, and archives sit at the expensive end.

What an audit trail actually looks like

In the projects we build, the audit trail is a data structure, not a policy document. A simplified record for a single extracted fact looks like this:

  • Claim: person X appears as a signatory on document Y.
  • Evidence: image region on page 3 of scan Y-003.tif; OCR text of that region; model and version that produced it; character-level confidence.
  • Retrieval: the query and passage set that surfaced this document, so the "why did the system show me this?" question always has an answer.
  • Verification: unverified / verified-by-human (who, when) / disputed.

The important property is that the chain is machine-checkable. Before anything is exported — a report, a finding aid, a batch of catalog records — a validation pass walks every claim back to its evidence and refuses to export anything orphaned. In a RAG-based research system this is straightforward to enforce, because retrieval already operates on identified passages; the discipline is in never letting prose bypass the mechanism.

Citation discipline for the human in the loop

The machinery only helps if the humans use it. Three working rules we apply, and recommend to any team adopting AI research tools:

Cite the source, never the tool. The AI is an instrument, like a microfilm reader. The citation in a footnote points to the archival item; the methodology section discloses that AI-assisted search and transcription were used, with tool names and versions. This matches the direction academic publishers and style guides have been converging on, and it is also simply accurate: the authority rests with the document.

Verify at the point of use. It is tempting to "verify" by re-reading the AI's transcription. Verification means opening the image of the page. In our memoir annotation work, every annotation that made it into the final apparatus was checked against the scan, not against the intermediate text.

Record the negative space. When the system searched for something and found nothing, log that too — the corpus version, the query, the date. "Not found in the collection as digitized in 2026" is a real scholarly statement, and it is only defensible if the search itself is documented.

Why this is becoming a requirement, not a preference

Institutions are formalizing exactly these expectations. Grant funders increasingly ask how AI-produced metadata will be flagged and reviewed. Journals ask authors to disclose AI assistance and to certify that citations were human-verified. Archives adopting AI cataloging distinguish machine-generated records from curated ones so that future staff — and future, better models — know which records to revisit.

The pattern behind all of it: AI turns research from a scarce activity into an abundant one, and abundance shifts the bottleneck to trust. The projects that will matter in five years are the ones whose every claim can still be walked back to a page.

That is a pipeline property. It has to be built in from the first day — retrofitting provenance onto a finished project is somewhere between painful and impossible.

If you are planning an AI-assisted research or cataloging project and want verifiability designed in from the start, that is precisely what we do.

Frequently Asked Questions

Can AI-generated research findings be cited in academic work?

Not directly. What can be cited is the primary source the AI helped locate — after a human has opened it and verified the passage. A defensible workflow treats every AI output as a lead, not a finding, and records both the source location (file, page, line) and the verification step.

What is an audit trail in AI-assisted research?

A record that links every claim in the final output back through each processing step to the original scan: which document, which page region, which OCR pass, which model version, what confidence score, and who verified it. If any link in that chain is missing, the claim cannot be defended.

How do you stop a language model from inventing citations?

Structurally, not by prompting. Retrieval-augmented generation (RAG) constrains the model to quote from retrieved passages, and the pipeline attaches the passage identifiers mechanically — outside the model — so a citation can never point to a document the system did not actually retrieve.

Do archives need audit trails if the research is internal?

Yes, for a practical reason: catalog records outlive the people and tools that made them. A record flagged as machine-generated with its confidence score can be revisited when models improve; an unflagged one silently becomes institutional truth, mistakes included.