AI vendor due diligence checklist: evidence first

Oryx Centre for Strategic Futures is a two-person research initiative founded 2 October 2026. It is a Udyam-registered micro enterprise (UDYAM-UP-56-0189301) — NOT a registered organisation and NOT a registered non-profit. Every number below from Oryx’s own work is labelled as a pilot result, not a universal rate.

A vendor demo shows answers. Due diligence asks a different question: what evidence stands behind those answers, and what happens when the evidence fails? This checklist is for the buyer who has to sign — or refuse to sign — on the basis of that answer.

It is written so a reader who never contacts Oryx can use it in full. Work through it with the vendor's own proposal, demo materials, and draft contract in front of you, and keep the completed checklist with the contract file.

What due diligence means here

Due diligence is not a reputation check. A well-known vendor can still sell a claim with no source behind it, and an unknown vendor can still document its claims properly. The check here is narrower and more useful: for each material claim, is there evidence you can open, does that evidence actually support the claim, and is someone named as responsible when it does not?

A claim is a checkable statement plus a source that should support it. Slogans — "accurate", "trusted", "enterprise-grade" — are not claims until they point at evidence. If a statement cannot be tied to evidence, treat it as marketing, not as a basis for signing.

This is not legal advice, and it does not replace your own legal, security, or procurement review. It is the evidence half of due diligence: the part most buyers skip because the demo looked convincing.

The checklist

A. Claims and evidence

• ☐ Every material claim in the proposal and demo is listed — not just the headline claims.
• ☐ For each claim, the vendor has supplied its source in writing: a document, dataset, evaluation, or reference you can open.
• ☐ Each source resolves: links, DOIs, and documents are real and retrievable, not plausible-looking references.
• ☐ Each source actually supports its claim, marked supports / partially supports / does not support. A real source saying something else is a failed check, not a pass.
• ☐ The version and date of the evidence match the version being sold. Evidence for an older model does not support claims about the current one.
• ☐ Evaluation figures, where claimed, state what was measured, on what inputs, and by whom — not just a percentage with no method behind it.

Why the source check matters is visible in published pilot evidence, labelled as pilot evidence and not a vendor test or a universal rate: in Oryx's published pilot audit of 66 AI-generated citations (Paper 2, study 1), 31 citations (47%) had broken links, 7 (11%) mismatched the claim they were attached to, and 11 (17%) could not be verified — only 5 both resolved and fully supported the claim. The point is not that every vendor fails at that rate. It is that "looks right" is a weak test, and due diligence that skips the resolve-then-support check has skipped the part most likely to fail.

B. The product as it will actually run

• ☐ The exact product, model version, and configuration being bought are named in writing — not just a product family name.
• ☐ Anything the demo showed that is not in the purchased version is listed as excluded, in writing.
• ☐ Known limitations are stated by the vendor, not discovered by you after signing: what the system does badly, and what inputs it handles poorly.
• ☐ Subcontractors and third-party components are disclosed where they affect the output: if another provider's model, data, or service sits inside the product, it is named.
• ☐ How the system will be monitored over time is described: models and sources change, so a check that passed at signing can fail silently later unless re-checking is built in.

C. Data handling

• ☐ What data you supply, what data the vendor retains, and what data is used for further training are each stated separately — a single "we handle data securely" sentence does not answer any of the three.
• ☐ Retention and deletion are dated: how long your data is kept, and by when it is returned or deleted at the end of the engagement.
• ☐ Who can access your data inside the vendor's organisation, and under what controls, is stated.
• ☐ If your data leaves your jurisdiction or passes through a subcontractor, that route is disclosed before signing, not after.

D. Human review and accountability

• ☐ A named human on the vendor's side is responsible for the output's evidence quality — not "the team" and not "the system".
• ☐ A human review step exists before consequential outputs reach you, and the reviewer can delay or reject an output.
• ☐ A named human on your side accepts or rejects the delivered work, and that acceptance is recorded in writing.
• ☐ When evidence fails after signing, the remedy is written in advance: correction, re-check, or termination — not "we will look into it".

E. Failure, remedy, and exit

• ☐ What happens when a claim proves false or a source cannot be verified is written before signing, with a timescale.
• ☐ Service levels, where promised, are measurable and tied to a remedy — a promise with no remedy is a hope, not a term.
• ☐ Exit is clean: you keep your data, your outputs, and your check record whatever the reason for ending, and there is no obligation to continue into a larger contract.
• ☐ Continuation beyond the initial term is a separate decision with separate terms — the agreement does not silently roll into a subscription or retainer.

F. Record-keeping

• ☐ The completed checklist, with sources and verdicts per claim, is kept with the contract file.
• ☐ The vendor's written answers to your questions are kept in the same file, so a later dispute is settled by the record, not by memory.
• ☐ A re-check date is set: at least one point after signing when the evidence check is re-run on the live product, not just on the demo.

How to score the result

Score each section, not just the vendor as a whole:

• Pass: every item in the section is met, with evidence on file.
• Conditional: named gaps, each with a fix and a re-check date. A conditional is not a pass, and it does not become one by waiting.
• Reject: any section fails on a material claim, the evidence check cannot be completed at all, or the vendor will not put a material answer in writing.

Record the section scores and the overall verdict in one page, attach it to the contract file, and keep it. Six months later, that page is the difference between "we checked them" and "we think we checked them".

Red flags

• No sources offered for material claims, or sources that do not resolve.
• Citations that point at real documents saying something else.
• "Trust us" offered in place of evidence, or evaluation figures with no method.
• No named human acceptor on either side.
• Data retention, training use, or subcontractors left vague.
• Evidence that cannot be re-checked after signing.
• No written remedy for failed evidence, and no clean exit.

Any one of these on a material claim is enough to hold signing until it is fixed — the checklist is not a scoring exercise where a good demo compensates for missing evidence.

Questions to send a vendor (copy-paste list)

1. For each material claim in your proposal, what is the source, and can I open it?
2. Does each source fully support its claim, partially support it, or not support it — and who marked it?
3. Which exact product version and configuration am I buying, and what did the demo show that is excluded?
4. What are the known limitations of this version, in writing?
5. What data of mine do you retain, what do you use for training, and by when is it deleted at exit?
6. Who inside your organisation can access my data, and which subcontractors touch it?
7. Who is the named human responsible for evidence quality, and can they delay or reject an output?
8. What is the written remedy when a claim proves false or a source cannot be verified?
9. How is the system re-checked over time as models and sources change, and when is the first re-check?
10. What do I keep if we exit, and does continuation require a separate agreement?

Sources and further reading

• Oryx Working Paper v3, From Answers to Auditable Intelligence (Zenodo DOI 10.5281/zenodo.23718472) — the six-layer framework behind the evidence, human-gate, and monitoring checks above.
• Paper 2 v2, Trust, But Verify (Zenodo DOI 10.5281/zenodo.23731235) — the published pilot citation audit quoted in section A.
• Oryx's published guide, "How to check an AI vendor's claims before you sign" (oryxai.in) — the six-step claim check this checklist builds on.