Evidence label: deterministic synthetic model. This page runs a small JavaScript model of a structured-extraction endpoint. For each of 300 synthetic requests it builds the token sequence of a JSON object with the chosen number of fields, then simulates a model that occasionally slips on syntax, mismatches the schema, adds prose, or meets an input with no value for a numeric field. Depending on the decoding mode, a mask removes some of those failures, forces values, or does nothing, and an illustrative latency model adds grammar compilation, mask work per token and optional jump-forward. Nothing here runs a real model; all numbers are calculated estimates.
Play the lab
How to play
Choose the decoding mode and mask backend, decide whether numeric fields may be null, toggle jump-forward, and set the number of fields, max_tokens and temperature. Every change re-runs the same seeded requests, so differences come from your change. Events make the model chatty, send a new schema with every request, remove facts from a quarter of inputs, or make string fields six times longer.
Missions unlock in order; each loads a broken setup. Stars reward lower latency or more correct fields once the goal is met.
What the model assumes
Per-token syntax slips occur with probability 0.0002 + 0.0008 × T², schema mismatches with 0.002 + 0.004 × T per field, and each field is semantically right with probability 0.995 − 0.02 × T². Prose before the JSON appears in 2% of answers (40% with the chatty event). In schema mode, slips and mismatches are masked away, and a missing numeric fact becomes an invented number unless the field is nullable. Latency is 120 ms prefill, 14 ms per model-generated token, a small cost per forced token under jump-forward, mask cost per token and, for schema mode, compile time on the first request or on every request with the dynamic-schema event. Backend costs are illustrative.
What it cannot tell you
Real error rates depend on the model, prompt and schema, and real mask costs depend on the engine, the vocabulary and whether mask work overlaps GPU execution. The lab does not model token-boundary effects or quality loss from reasoning under format constraints. Use it to see which failure each technique removes and which it creates, then measure your own endpoint.
Read the full explanation
- Constrained Decoding: How JSON Schema Mode Masks Invalid Tokens (companion article)
- Sampling Settings That Fix Real Failures: Temperature, Top-p and Min-p
- Logprobs and Calibration: Measuring When a Language Model Is Guessing
All AI in Production labs · Lab catalog
- Efficient Guided Generation for Large Language Models (Willard and Louf, 2023)
- XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models (Dong et al., 2024)
- SGLang: Efficient Execution of Structured Language Model Programs (Zheng et al.)
- Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models (Tam et al., 2024)
- JSON Schema specification
- vLLM documentation: engine arguments
Views are my own and do not represent any employer, past or present.