Why it matters
Medusa avoids draft model deployment. Growing adoption.
Advertisement
The architecture
Target model + K medusa heads.
Each head predicts token at position +N.
Verify accept prefix.
Advertisement
How it works end to end
Heads: N MLPs on final hidden state.
Training: fine-tune heads on model outputs.
Tree attention: multiple candidates.