Evidence label: deterministic synthetic model. This page runs a small JavaScript model of citation verification in RAG. It generates synthetic answers made of sentences and atomic claims, each citing one of five passages, with hidden labels for supported claims, claims with no source, claims citing the wrong passage and claims with a changed number. A verifier scores each verification unit against its citation with a chosen separation and noise, a threshold decides pass or fail, and a failure is dropped, re-cited or regenerated. Nothing calls a model; every number is a calculated estimate from a synthetic model.
Play the lab
How to play
Choose a verifier and the unit it checks (whole answer, sentence or atomic claim), set the support threshold τ and pick what happens when a unit fails. Every change re-runs the same 320 answers with the same random draws, so differences come from your settings. Watch the two key tiles together: unsupported claims shipped (lower is better) and true claims kept (higher is better). The histogram shows why they trade off.
Missions unlock one at a time; Load mission setup starts from the broken configuration. Stars reward the faster, cheaper or more complete fix. Toggle loose citations or numeric-heavy answers at any time to see how robust your setup is.
What the model assumes
By default 8% of claims have no supporting passage, 5% cite the wrong passage although another passage in the prompt supports them, and 3% change a number; the events raise mis-citation to 20% and number drift to 14%. Verifier scores are logistic in a latent value of plus or minus half the separation with unit Gaussian noise; separations are 1.4 (lexical), 3.0 (NLI) and 3.6 (LLM judge), and number drift is easier or harder for each. Sentence and answer units average their claims' latent values. Decomposition into claims adds 180 ms, NLI checks cost 25 ms plus 3 ms per pair, an LLM judge call 400 ms plus 30 ms per pair, and regeneration 650 ms. Re-attaching a citation requires τ + 0.15. Regeneration fixes an unsupported claim 60% of the time.
What it cannot tell you
Real verifiers are not Gaussian scorers: their errors cluster on particular phrasings, long passages and domain jargon, and an LLM judge can be swayed by confident wording. Decomposition can itself split or merge facts incorrectly. The model also says nothing about whether the passages are correct or current. Use the lab to understand how verification unit, threshold, verifier strength and failure handling interact, then calibrate on labelled claims from your own system.
Read the full explanation
- Grounded Citations in RAG: Verifying Every Claim Against Its Source Passage (companion article)
- RAG Failure Triage: Separating Retrieval Misses From Generation Errors
- Lost in the Middle: Ordering and Packing Retrieved Context for Long Prompts
All AI in Production labs · Lab catalog
- Enabling Large Language Models to Generate Text with Citations (ALCE, Gao et al., EMNLP 2023)
- Measuring Attribution in Natural Language Generation Models (AIS, Rashkin et al.)
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (Min et al., EMNLP 2023)
- RAGAS: Automated Evaluation of Retrieval Augmented Generation (Es et al.)
- TRUE: Re-evaluating Factual Consistency Evaluation (Honovich et al., NAACL 2022)
Views are my own and do not represent any employer, past or present.