Why it matters

Without CI, agent quality drifts silently. Understanding CI patterns enables reliable delivery.

Advertisement

The architecture

Build: gradle/maven build. Static checks.

Unit tests: fast, mocked. Run every PR.

Integration: real LLM. Sampled or full per PR.

Agent CI pipelineBuild + staticcompile + lintUnit testsmocked, fastIntegration + evalreal LLM, cost-managedCache eval results by prompt hash; skip if prompt unchanged
CI stages.
Advertisement

How it works end to end

Eval on PR: run eval set; report metrics; block merge on regression.

Cost management: sample eval on PR, full on main. Cache results.

Nondeterminism: run multiple samples for flake resistance.