Core concept
Llama 3.1 8B distilled from 405B. Phi from GPT-4. Common pattern.
Advertisement
How it works
Compute cost to train student. Not trivial. But amortized over massive inference.
Advertisement
Trade-offs + gotchas
Product deployment. Move from 70B to 8B for order-of-magnitude cost savings.