Why it matters
CoT is what turns LLMs from 'usually right' to 'often right' on reasoning tasks. Understanding when to apply it is essential for building reliable systems.
The architecture
Zero-shot CoT: 'Let's think step by step.' appended to prompt. Model generates reasoning trace before final answer.
Few-shot CoT: examples show explicit reasoning between input and answer. Model learns to produce similar reasoning.
How it works end to end
Why it works: reasoning tokens give the model 'thinking room'. It can build up intermediate results before committing to a final answer.
Self-consistency: sample multiple CoT traces, take majority answer. Reduces errors from single-shot reasoning.
Tree-of-thoughts: explore multiple reasoning branches, backtrack from dead ends. More powerful, more expensive.
Modern reasoning models (o1, DeepSeek R1) train specifically for CoT, doing it internally without prompt engineering.