Why it matters

Evaluation shapes quality iteration. Understanding shapes design.

Advertisement

The architecture

Evaluator.evaluate(request).

LLM-as-judge under hood.

Chain into CI or advisor.

Evaluator flowEvaluatorabstractionJudge LLMunder hoodScorepass / fail / valueCombine multiple evaluators; integrate with tests + CI
Evaluator.
Advertisement

How it works end to end

RelevancyEvaluator: is answer relevant?

FactCheckingEvaluator: is answer supported?

Custom via LLM prompt.