Why it matters
Wrong temperature produces wrong-feeling output: too rigid for creative tasks, too random for factual ones. Getting it right matches output to use case.
Advertisement
The architecture
Softmax with temperature T: probability of token = exp(logit / T) / sum. Higher T flattens; lower T sharpens.
T=0 (greedy): always pick highest-probability token. Deterministic.
Advertisement
How it works end to end
T=0 gives repeatable results. Use for factual answers, code generation, structured output.
T=0.3-0.5 balances consistency with variation. Good default for chat.
T=0.7+ introduces creative variation. Good for brainstorming, writing.
Very high T (1.5+) can produce incoherent output.