Why architecture matters here
The core problem is attribution. LLM output quality is a function of prompt, model version, sampling parameters, and input distribution — four variables that all drift. When a support-bot's escalation rate doubles, the team without a registry spends the incident reconstructing what was live: was it Tuesday's prompt tweak, the provider's silent model update, or a new customer segment? The team with a registry reads it off the output telemetry: every response carries prompt=support_triage@v41, model=claude-sonnet-5, temp=0.3, and the regression correlates exactly with v41's canary window. Attribution converts week-long mysteries into five-minute diffs.
The second force is velocity with safety. Prompt iteration is the highest-leverage tuning loop in an LLM product — teams that iterate daily outperform teams that iterate monthly — but raw speed without gates means every edit is a production experiment on live customers. The registry's eval gate makes the safe path the fast path: a candidate version runs against golden sets in minutes and either earns promotion or returns with specific failures. Crucially this also enables non-engineers: product managers and domain experts can author prompt changes through the same gated pipeline, because the gate — not code review of a string diff — is what protects production.
Third, prompts multiply. A real product is not one prompt; it is dozens — routing, extraction, generation, summarization, safety-review — composed in chains where one prompt's output format is the next one's input contract. Unmanaged, these drift independently until a 'harmless rewording' upstream breaks a parser two hops downstream. A registry gives the chain explicit contracts: variable schemas declare what each template needs, output-format assertions live in the eval sets, and version pinning means a chain in production is a reproducible tuple of specific versions, not whatever strings happened to be live at request time.
The architecture: every piece explained
Top row: the supply side. Authoring keeps prompt sources in git — templates plus a manifest declaring the variable schema (names, types, required/optional), expected output format, and the eval suites that gate promotion. Authors work in a playground wired to the same template engine as production (parity here prevents a whole category of render-time surprises). Publishing creates a record in the registry store: an immutable version — content-hashed, monotonically numbered — with metadata (author, changelog, parent version, linked eval results). Immutability is non-negotiable: v41 means the same bytes forever, which is what makes telemetry attribution and rollback meaningful.
The eval gate runs every candidate against its declared suites: golden input/output sets scored by exact-match or embedding similarity for extraction tasks, LLM-as-judge rubrics for open-ended generation, format validators for structured output (does every response parse against the JSON schema?), safety probes (injection attempts, forbidden-content bait), and regression comparisons against the currently promoted version on the same inputs. Results attach to the version permanently. The gate is policy-configurable per prompt: a marketing-copy prompt might promote on judge-score parity, while a medical-triage prompt requires human review plus a zero-regression bar. Promotion then moves an environment label — prod points at v41 — and labels, not versions, are what runtimes request.
Middle row: the demand side. The runtime resolver is a thin client library: it asks the registry for support_triage@prod, receives the pinned bundle, and caches it with a short TTL plus push invalidation. The cache is also the availability story — resolvers serve stale-on-error and persist a last-known-good snapshot, so a registry outage degrades to 'no new promotions' rather than 'no prompts'. The template engine renders variables into the template with strict-mode guards: undeclared variables are errors, missing required variables are errors, and user-supplied values are fenced per the template's escaping rules — render-time strictness is your last defense against schema drift. The rollout controller sits at resolution: it can answer a prod lookup with v41 for 90% of sessions and v42 for a 10% canary, or run a proper A/B experiment with session-sticky assignment feeding the experimentation platform.
The often-missed piece: model config travels with the version. A prompt is tuned against a specific model and parameter set; promoting the text while the model choice lives in a different config system reintroduces the attribution problem. The registry bundle pins template + model + temperature + max tokens + stop sequences as one promotable unit. Bottom row: telemetry stamps every LLM call with the resolved version tuple, flowing into logs, traces, and the eval platform; the feedback loop mines production traffic for failures (thumbs-down, parse errors, escalations) and turns them into new golden-set entries — the mechanism by which eval suites grow teeth over time instead of fossilizing.
End-to-end flow
Follow one change. Step 1 — author. A PM notices the support-triage bot over-escalates billing questions. In the playground she edits support_triage's instructions, tests against a handful of saved cases, and publishes: the registry creates v42 (parent v41, changelog 'clarify billing-tier escalation criteria'), content-hashed and immutable. Step 2 — gate. CI runs the declared suites: 240 golden triage cases (v42 fixes 11 of 14 known billing over-escalations, no new misses), the format validator (100% parse), the injection probe set (clean), and the head-to-head judge comparison with v41 (win rate 58/22/160 win/loss/tie). The gate passes; v42 is promotable.
Step 3 — staged promotion. She moves staging to v42; the staging bot picks it up on the next cache refresh (~30s) and the team dogfoods for a day. Then prod promotion starts as a canary: the rollout controller serves v42 to 10% of sessions, sticky by session ID. Step 4 — watch. Dashboards split every metric by prompt version, which telemetry stamping makes a groupby rather than a project: escalation rate 14%→9% in the canary cohort, parse errors flat, latency flat (v42 is 40 tokens shorter — a small win), thumbs-down flat. After 48 clean hours the controller ramps 25%→100%, and prod now means v42 everywhere.
Step 5 — the surprise and the rollback. A week later the model provider ships a point update, and v42's carefully worded escalation criteria interact badly with it — escalations spike at 3 a.m. The on-call engineer's dashboard shows the spike is uniform across prompt versions (ruling out a prompt change) but correlated with the model update; because model config is pinned in the registry, the mitigation is a registry operation, not a code deploy: repoint prod's model pin to the prior model snapshot — a one-command rollback that propagates through cache invalidation in under a minute. Step 6 — close the loop. The 3 a.m. failing conversations are minted into golden-set entries tagged with the incident; v43 will be gated against them forever. The registry did not prevent the incident — it made the diagnosis a groupby and the fix a label move.