Why it matters

LLM-as-judge scales quality eval. Human spot-check mitigates bias.

Advertisement

The architecture

Judge prompt + rubric.

Score outputs.

Human spot-check.

LLM-as-judgeJudge promptwith rubricScore outputs1-5 or pass/failHuman spot-checkbias mitigationWidely used; MT-Bench, AlpacaEval used at scale; needs bias checks
LLM-as-judge.
Advertisement

How it works end to end

MT-Bench, AlpacaEval.

Rubric-driven.

Human spot-check.