Why it matters
SLMs are where the industry is heading for productionized LLM applications. Cheaper inference, lower latency, on-device deployment. Understanding them shapes cost-effective AI architecture.
The architecture
Definition: usually 1B-13B parameters. Runs on consumer GPU (24GB) or even CPU. Trained on high-quality data with efficient methods.
Capabilities: rival much larger models on focused tasks; underperform on complex reasoning and world knowledge.
How it works end to end
Training approach: high-quality curated data (not just web scrape), longer training on smaller model, instruction tuning, distillation from larger models.
Deployment: fits on 8GB (quantized) to 24GB GPU. On-device on modern phones. Sub-second latency at reasonable throughput.
Task fit: excellent for narrow tasks (classification, extraction, focused generation). Falls behind on open-ended reasoning and rare-knowledge queries.