Why it matters

Untested agents drift. Understanding testing enables confidence.

Advertisement

The architecture

Unit: mock model interface. Verify tool selection, output format.

Integration: real model. Confirm end-to-end works.

Eval: test set + metrics.

Agent testing pyramidUnit testsmocked LLMIntegration testsreal LLMEval harnessquality metricsSmall integration test set for cost; larger unit set for coverage
Testing types.
Advertisement

How it works end to end

Mocked LLM: return predetermined responses per prompt pattern. Fast + deterministic.

Integration cost: full LLM calls; keep set small.

Eval: run on every deploy; block on quality regression.

Trace-based: analyze agent trajectories, not just outputs.