LLM Distillation Data

Why it matters

Distillation data enables small models. Understanding shapes strategy.

The architecture

Teacher LLM generates responses / reasoning.

Student LLM trains via SFT + distillation loss.

How it works end to end

Response generation + optional reasoning.

Soft target logits when available.

Domain coverage.