AI governance evidence checklist: prove every claim

Oryx Centre for Strategic Futures is a two-person research initiative founded 2 October 2026. It is a Udyam-registered micro enterprise (UDYAM-UP-56-0189301) — NOT a registered organisation and NOT a registered non-profit. Every number below from Oryx’s own work is labelled as a pilot result, not a universal rate.

AI governance usually fails in the same quiet way: a system is adopted on the strength of a demo and a set of confident statements, and nobody can later show what was claimed, what evidence supported it, or who accepted the risk. A governance evidence checklist exists to prevent that. It is not a policy document. It is a file you can open after a decision and use to reconstruct exactly what was known, and what was not, when the decision was made.

This page teaches the checklist itself. You need nothing from Oryx to use it.

What “evidence” means here

Evidence, for governance purposes, is narrower than marketing material. A claim is a checkable statement: it says something specific that could turn out to be true or false. A source is the document, dataset, link, or DOI that is supposed to support it. Governance evidence is the record linking the two, plus a named human who accepted whatever risk remained after the check.

Slogans are not claims. “Accurate”, “trusted”, and “enterprise-grade” say nothing checkable until they point at evidence. If a statement cannot be tied to a source, it does not belong in the evidence file — it belongs in a separate list of unproven assertions, labelled as such.

The checklist

1. Inventory every material claim

• ☐ Every material claim in the proposal, demo, or contract is listed — not just the headline claims.
• ☐ Each claim is written in checkable form: what exactly is asserted, about which system, version, and use case.
• ☐ Slogans are separated from claims and marked as unproven, not quietly counted as evidence.
• ☐ Claims made verbally in a demo are written down too; a claim that exists only in a meeting has no evidence file.

What good looks like: a stranger reading the inventory could say, without asking anyone, what was promised and what was merely implied.

2. Check that each source resolves

• ☐ For each claim, the source is named in writing: a link, DOI, document, or dataset identifier.
• ☐ The source is actually retrievable — the link opens, the DOI resolves, the document can be obtained.
• ☐ A source that cannot be retrieved is recorded as a failed check, not carried forward as if it had passed.

This step matters more than it sounds. In Oryx’s published pilot audit of 66 AI-generated citations (Paper 2, study 1 — a pilot, not a universal rate), 31 citations (47%) had broken links. A governance file that never tested whether its sources open would have filed those as evidence.

3. Check that the source supports the claim

Resolving is not enough. A real source saying something else is a failed check, not a pass.

• ☐ For each claim, the relevant passage of the source has actually been read — not just the title or abstract.
• ☐ Each claim is marked: supports / partially supports / does not support.
• ☐ A “partially supports” mark names the gap: what part of the claim is unsupported, and what would close it.
• ☐ A “does not support” mark is never softened into a pass because the source is real or reputable.

In the same published pilot audit, 7 of the 66 citations (11%) mismatched the claim they were attached to, and 11 (17%) could not be verified at all — only 5 both resolved and fully supported the claim. The lesson for governance is the method, not the rate: resolve first, then support, then record. Skipping the second step turns a real bibliography into a false sense of assurance.

4. Check version, date, and scope

• ☐ The evidence matches the version being adopted, not an earlier or later version.
• ☐ The evidence is current for the use case: dated material is flagged where newer evidence exists.
• ☐ The scope matches: evidence from one setting (one language, population, or task) is not filed as evidence for a different setting without that mismatch being named.
• ☐ Model, dataset, and system versions are recorded alongside the claims they support, so a later update can be re-checked rather than assumed still valid.

5. Name the human decision gate

Governance without a named human is a filing cabinet, not accountability. This step draws on the human decision gate in Oryx’s working paper framework (v3, six layers), where consequential judgements pause for an accountable human to decide.

• ☐ One named person accepts the residual risk for each material claim or claim group. “The team will review it” is not a name.
• ☐ The acceptor is independent of the party that produced the claim where that is possible; where it is not, the conflict is recorded.
• ☐ The acceptor’s decision — accept, accept conditionally, or reject — is recorded in writing with reasons.
• ☐ Conditional acceptance names the conditions, the owner of each fix, and a re-check date. A condition without a date is not a control.

6. Record failure, remedy, and re-check rights

• ☐ What happens when evidence fails is written before signing: fix and re-check, price or scope adjustment, or exit.
• ☐ Re-check rights after signing are stated: can the buyer re-run the evidence check when the system updates, and what access does that require?
• ☐ The remedy for a claim that turns out false after adoption is named, not left to a future argument.

7. Keep the record with the decision

• ☐ The completed checklist, the source list, the support marks, and the named acceptances are filed with the contract or decision record.
• ☐ The file is dated and versioned, so a later reviewer can tell which evidence supported which decision, and when.
• ☐ Changes after adoption — new versions, new claims, new evidence — are appended as new dated entries, not edited over the original record.

Red flags

• No sources offered for material claims, or sources offered only on request after signing.
• Sources that do not resolve, or citations that do not support the claim they are attached to.
• “Trust us” offered in place of evidence, or evidence that cannot be re-checked after signing.
• No named human acceptor — accountability assigned to a committee, a process, or a tool.
• Evidence from a different version, date, or setting filed without the mismatch being named.
• A governance file that exists only as a slide deck: no claim inventory, no support marks, no record.

What to do with the result

Pass: every material claim has a resolving source that supports it, and a named human has accepted the residual risk in writing. File the record and set a re-check date for the next version change.

Conditional: named gaps, each with a fix, an owner, and a re-check date. A conditional is not a pass, and it does not become one by waiting.

Reject: any material claim fails on evidence, or the evidence check cannot be completed at all. Record the reason; do not let the decision proceed on the strength of the remaining claims without re-scoping what is actually being adopted.

Questions to send before adoption (copy-paste list)

1. For each material claim, what is the source, in writing, and can I open it myself?
2. Which passage of each source supports the claim, and where does support stop?
3. Does this evidence match the exact version and use case we are adopting?
4. Who, by name, accepts the residual risk for each claim group?
5. What happens if a claim’s evidence fails after we sign?
6. Can we re-run this evidence check when the system updates, and what access do we need?
7. Where will the completed checklist be filed, and who keeps it current?

Sources and further reading

• Oryx Working Paper v3, From Answers to Auditable Intelligence (Zenodo DOI 10.5281/zenodo.23718472) — the six-layer framework behind the claim-traceability and human-gate checks above.
• Paper 2 v2, Trust, But Verify (Zenodo DOI 10.5281/zenodo.23731235) — the published pilot citation audit quoted in sections 2 and 3.

---

Oryx AI offers claim-level evidence audits for research drafts and vendor claims — ₹7,500 per pilot report, payable only on acceptance, currently invite-only beta (contact: amartya@oryxai.in).