Split by session_id hash

Deterministic per user — same user always gets the same variant. Prevents thrash.

Advertisement

Config-driven routing

String variant = experimentClient.getVariant("prompt_v2", ctx.userId());
BaseLLM llm = variant.equals("treatment") ? newLLM : oldLLM;
Advertisement

Metrics to compare

Task completion rate. User satisfaction (thumbs up). Cost per session. Latency.