Why architecture matters here
SLM eval fails when it only uses public benchmarks (gamed), skips human eval, or ignores safety. Architecture matters because a suite that covers task + safety + cost is what convinces stakeholders.
The architecture: every piece explained
The top strip is measurement. Model candidate. Benchmark suite — MMLU, HELM, etc. Task-specific suite — the actual product tasks. Human eval — small but critical samples.
The middle row is quality. Safety eval. Regression harness — golden set. Latency + cost per token. Report — verdict + tradeoffs.
The lower rows are ops. CI integration per PR. Comparison vs baseline + larger models. Ops — governance + versioning.
End-to-end flow
End-to-end: 7B SLM candidate. Benchmarks show 88% of 70B on target tasks. Human eval on 200 samples confirms quality. Safety eval passes. Latency 3x faster, cost 5x lower. Report recommends ship. CI runs same eval on future PRs.