Human review gate AI: what a human must check

Oryx Centre for Strategic Futures is a two-person research initiative founded 2 October 2026. It is a Udyam-registered micro enterprise (UDYAM-UP-56-0189301) — NOT a registered organisation and NOT a registered non-profit. Every number below from Oryx’s own work is labelled as a pilot result, not a universal rate.

An AI system can draft a report in minutes. That speed is the risk: a confident-sounding draft can reach a decision-maker before anyone has checked whether its claims are true. A human review gate is the pause placed between the draft and the decision — a named person, with time and authority to stop it, checking the output before it is used.

This page teaches what that gate should check. You need nothing from Oryx to use it.

What a human review gate is — and what it is not

A gate is a decision point, not a disclaimer. It is a gate when all four of these are true:

1. A named human is responsible. Not “the team” and not “the system” — one person whose name is on the check.
2. The check happens before use. Reviewing a report after it has already informed a decision is not a gate; it is an apology.
3. The reviewer can stop it. If the reviewer cannot delay, reject, or send the work back, the gate is decoration.
4. The check is recorded. What was checked, what failed, and what was changed — in writing, kept with the report.

A footer that says “AI-generated, please verify” is not a gate. Neither is a quick skim for typos, nor an approval click by someone who did not read the evidence. If none of the four conditions holds, say so plainly: the output is unreviewed.

Where gates belong

Not every AI output needs the same gate. Put a gate wherever the output is consequential — that is, wherever acting on a wrong answer would cost something real: a wrong citation in a submitted paper, a wrong figure in a board paper, a wrong claim in a vendor proposal, a wrong summary in a policy brief.

A practical rule: gate the outputs you would be embarrassed to defend in public, and gate anything that will be quoted, filed, or sent onward without its draft status attached. Low-stakes uses — brainstorming, first-draft phrasing, summarising a document you already know — still benefit from a spot check, but they do not need a formal gate.

In Oryx’s own published framework, human decision gates are one of six layers: the question is framed, evidence is gathered, claims are traced to their sources, outputs are challenged, consequential judgements pause for an accountable human, and the system is re-checked over time as models and sources change. The gate is layer five, not the whole method — it catches what the earlier layers missed, and it only works because the earlier layers leave a trail to check.

The five checks a reviewer performs

A reviewer does not re-read the whole draft for style. They check the five things most likely to be wrong:

1. Can every material claim be traced to a source?

A material claim is any statement the decision depends on. For each one, the reviewer asks: is there a named source, and can I reach it? An unsourced claim is not “probably fine” — it is quarantined: flagged in plain sight, not quietly deleted, until it is sourced or dropped.

2. Does the source actually support the claim?

A real source saying something else is a failed check, not a pass. The reviewer reads the relevant passage and marks each claim: supports, partially supports, or does not support. This is the check most often skipped, because a plausible-looking citation feels like evidence. It is not — it is only a pointer, and pointers can point at the wrong thing.

3. Is the citation real?

AI-generated citations fail this test often enough that it deserves its own step. In a published pilot audit of 66 AI-generated citations, 31 (47%) had broken links, 7 (11%) mismatched the claim, 11 (17%) could not be verified, and only 5 (8%) both resolved and fully supported the claim (Paper 2, study 1 — a pilot, not a universal rate). The reviewer’s job here is mechanical: follow the link or DOI, confirm the title, authors, year, and journal match, and treat a reference that appears nowhere as fabricated until proven otherwise.

4. Is anything important missing?

AI drafts fail by omission as well as by error: a counter-argument never mentioned, a newer source never consulted, a caveat smoothed away. The reviewer asks what a sceptical reader would raise, and whether the draft answers it. Adversarial review — hunting for counter-evidence and weak sourcing — is a separate layer in Oryx’s framework for exactly this reason.

5. Who accepts the remaining risk?

No check removes all risk. After the first four checks, something always remains uncertain: a source behind a paywall, a figure from a single study, a claim that is plausible but not fully proven. The gate ends with a named human accepting that residual risk in writing — or refusing to, and sending the work back. Without that name, the risk is owned by nobody, which means it is owned by whoever acts on the report last.

What to record at the gate

A gate that leaves no record cannot be audited later, which defeats its purpose. Keep one short record per reviewed output:

• The output and its date, and the version of the system that produced it.
• The reviewer, by name, and the date of the review.
• Per material claim: the source, and the verdict (supports / partially supports / does not support / could not be verified).
• What was changed as a result of the review, and what was left uncertain.
• The name of the person accepting the residual risk.

That record travels with the report. If a reader, reviewer, or auditor later asks “who checked this?”, the answer is in the file, not in someone’s memory.

Common ways gates fail

• The reviewer is the author. Self-review catches typos, not reasoning errors. Where possible, the gatekeeper should not be the person who prompted the draft.
• The reviewer has no time. A gate with a five-minute slot for a forty-page report will pass everything. Size the gate to the output, or shrink the output.
• The reviewer cannot say no. If rejecting a draft has career consequences, or if the deadline makes rejection impossible, the gate will pass everything. That is a management problem, not a reviewer problem — fix the incentives, not the reviewer.
• The check is not recorded. An unrecorded check is indistinguishable, a month later, from no check at all.
• The gate checks style, not evidence. Grammar, formatting, and tone are the easiest things to check and the least important. A beautifully formatted report with fabricated citations is still a failed report.

A short routine for setting up a gate

1. List the outputs that are consequential enough to need a gate, and write the list down.
2. Name one reviewer per output type, and one deputy for when they are away.
3. Give the reviewer the five checks above, in writing, and the authority to delay or reject.
4. Keep the gate record with the report, in the same file or folder.
5. Re-check the gate itself occasionally: sample a passed report and re-run the five checks. If the sample fails, the gate needs fixing, not the sample.

Sources and further reading

• Oryx Working Paper v3, From Answers to Auditable Intelligence (Zenodo DOI 10.5281/zenodo.23718472) — the six-layer framework, including human decision gates.
• Paper 2 v2, Trust, But Verify (Zenodo DOI 10.5281/zenodo.23731235) — the pilot citation audit quoted above.