How to check AI citations in your paper

In short: run a paper-level pass over the whole reference list before submission: inventory every citation, resolve every link and DOI, match each source against the record, check support for the claim, and log a verdict per citation — with free tools and a 30-minute routine.

If an AI tool helped draft your paper, the errors it introduced are probably not in your argument. They are in your reference list. A fabricated citation looks exactly like a real one on the page — same author format, same journal style, same confident DOI — and it will sit there quietly until a reviewer, an editor, or a reader follows the link and finds nothing, or finds a paper that says something else.

This guide is a paper-level pass: a single routine you run over the whole reference list before submission. It is not a per-citation deep dive — that is covered in Oryx’s companion guide, “How to check AI-generated citations in your own paper” — it is the pass that makes sure no citation in your paper escapes checking at all. You need no product and no paid tool to follow it: a browser, Crossref, OpenAlex, and about 30 minutes.

Start with the reference list, not the text

Most people check citations as they read their draft, one at a time, in the order they appear. That is how citations get missed: the ones in footnotes, in tables, in the methods section you stopped re-reading three drafts ago.

Start instead by extracting the full reference list into one place — a spreadsheet or a plain text file, one row per citation, with the in-text claim it supports noted beside it. Then mark each row: did an AI tool produce or suggest this citation, or did you find it yourself? AI-assisted citations get checked first and hardest, because that is where the failure rate concentrates. But check the whole list. A citation you typed by hand can still carry a wrong year or a dead link.

Your inventory is also your defence. If a reviewer questions a citation later, a one-line log per citation — citation, verdict, date checked — shows the check happened. Keep it with the paper, not in your head.

The three ways AI citations fail in a paper

In a published pilot audit of 66 AI-generated citations (Paper 2, study 1 — a pilot, not a universal rate), the failures fell into three kinds:

• Broken links — 31 of 66 (47%). The DOI or URL does not resolve. The paper cannot be found at the address given. This is the commonest failure, and the easiest to catch: following the link takes seconds.
• Mismatched — 7 of 66 (11%). The link resolves to a real paper, but the paper does not say what your sentence claims it says. A real source cited for a claim it does not make is a failed check, not a pass.
• Could not be verified — 11 of 66 (17%). The citation looks complete but cannot be traced to any real record — no matching title, authors, or DOI in Crossref, OpenAlex, or Google Scholar. A perfect-looking reference that appears nowhere is fabricated until proven otherwise.

Only 5 of the 66 both resolved and fully supported the claim. That is a pilot result from one audit, not a rate you should quote about AI tools in general. Its lesson for your paper is narrower and safer: do not trust a citation because it looks well-formed. Resolve it, match it, and read what it actually says.

The paper-level pass, in six steps

Run these over the whole reference list, in order. Do not skip ahead to fixing until the pass is complete — you want the full picture of how many citations fail before you start repairing.

1. Inventory every citation. One row per reference, with the claim it supports. Count them. If the count surprises you, that is the first finding: citations you forgot were there are citations nobody has checked.

2. Resolve every link and DOI. Click each one, or paste each DOI into the Crossref resolver. Mark “resolves” or “broken” for every row. A broken link is a failed citation, not a minor flaw — fix it or remove the citation before you go further.

3. Match the paper, not just the link. For each citation that resolves, check that the title, authors, year, and journal match what your reference list says. A link that leads to a different paper with a similar title is a mismatch, and it usually means the citation was assembled from fragments of real references.

4. Check support for the claim it carries. Read the passage the citation is supposed to support — the abstract at minimum, the relevant section where the claim is specific. Mark each citation: supports, partially supports, or does not support. “Partially supports” is a real verdict: it means the claim needs narrowing to what the source actually says.

5. Cross-check anything you cannot trace. For citations that do not resolve or do not match, search the title in Crossref, OpenAlex, or Google Scholar. If the paper exists under a different DOI or year, correct the reference. If it appears nowhere, treat it as fabricated: do not keep it in the hope that a reader will not check.

6. Log the verdict per citation. One line each: citation, verdict, date checked. The log is what turns a private tidy-up into a check you can show a co-author, a supervisor, or a reviewer.

What to do with the citations that fail

A failed citation has three legitimate endings, and no others:

• Replace it with a real source that supports the same claim. This is the usual fix for a broken link where the underlying paper exists.
• Re-source the claim — rewrite the sentence to say only what a source you have actually read supports. This is the fix for a mismatch: the claim was the problem, not the citation.
• Drop the claim. If no real source supports it, the sentence goes. A gap in your argument is honest and fixable; a fabricated citation is neither.

Never keep a citation you could not verify, and never cite a paper you have not at least located. “The AI gave it to me and it looked right” is not a defence a reviewer will accept.

Traps that only show up at paper level

Some failures are invisible when you check citations one at a time. They only appear when you look at the whole list:

• The same source cited two different ways. One entry says 2023, another says 2024; one spells the author differently. At least one of them is wrong, and a reader who checks both will trust neither.
• A secondary citation passed off as primary. Your reference list cites the original study, but the AI actually read a summary of it. If you have not read the original, cite the source you did read — or read the original.
• A preprint cited as peer-reviewed work. Preprints are citable, but they are not peer-reviewed. If your sentence implies peer review, the citation overstates its own authority.
• The wrong version or year. A paper with multiple versions — preprint, conference version, journal version — cited by the year of the wrong one. Check the version you actually mean.
• A real journal name on a paper that does not exist. Fabricated citations routinely borrow real journal names. The journal being real proves nothing about the paper.

A 30-minute pre-submission routine

Do this the day you submit, after the last edit:

1. Re-run step 2 — resolve every link and DOI — on the final version of the paper. Links break between drafts more often than you expect.
2. Re-check every AI-assisted citation against its log entry. If a sentence changed since the check, the check is stale: re-run steps 3 and 4 on that citation.
3. Read the reference list once, top to bottom, looking only for duplicates, inconsistent years, and entries that do not match their in-text citations.
4. Confirm the log is saved with the submission files.

Thirty minutes is the whole routine for most papers. The cost of skipping it is a failed citation found by someone else, after the paper is already out.

What this check cannot do

Checking your citations verifies the chain from claim to source. It does not verify the science: a claim can be faithfully supported by its source and still be wrong, if the source itself is wrong. It does not check claims you left uncited. And it does not judge whether a source is the best source for the claim — only whether the source you cited says what you said it says. Those are separate checks, and worth running alongside.

Sources and further reading

• Paper 2 v2, Trust, But Verify (Zenodo DOI 10.5281/zenodo.23731235) — the published pilot citation audit quoted above.
• Oryx Working Paper v3, From Answers to Auditable Intelligence (Zenodo DOI 10.5281/zenodo.23718472) — the six-layer framework behind the claim-level check.
• Oryx’s guide, “How to check AI-generated citations in your own paper” (oryxai.in) — the five-step check for a single citation, which this paper-level pass applies across the whole reference list.
• Oryx’s guide, “How to check if an AI citation is real” (oryxai.in) — the real-or-fabricated check for a single suspicious reference.
• Oryx’s guide, “Claim-level evidence audit: check each claim” (oryxai.in) — the same resolve-then-support method applied claim by claim to a whole document.
• Oryx’s guide, “Auditable AI research: what makes research checkable” (oryxai.in) — the five properties that make any report, including an audit report, checkable.

Related: How to check AI-generated citations in your own paper — the five-step check for a single citation, which this paper-level pass applies across the whole reference list.

---

Oryx AI offers claim-level evidence audits for research drafts and vendor claims — ₹7,500 per pilot report, payable only on acceptance, currently invite-only beta (contact: amartya@oryxai.in).

Oryx Centre for Strategic Futures is a two-person research initiative (Udyam-registered micro enterprise) working on auditable AI.