Why architecture matters here
The reason placement matters is that training a deep network is fundamentally a problem of getting a usable gradient back to the early layers. In a stack of dozens of blocks, the gradient of the loss with respect to layer 1's parameters is a long product of Jacobians, one per intervening layer. If each factor shrinks the signal even slightly, the product vanishes exponentially with depth and the early layers stop learning; if each factor amplifies, it explodes and training diverges. The residual connection is the celebrated fix — it adds an identity term to each layer's Jacobian, so the gradient has a path of multiplying by (roughly) one all the way back. But whether that path stays clean depends entirely on what else sits on the residual stream, which is exactly what norm placement decides.
In post-norm, the computation is LayerNorm(x + Sublayer(x)): the normalization wraps the residual add, so it is on the identity path. On the backward pass, the gradient flowing to earlier layers must pass through every one of these LayerNorms, and LayerNorm's Jacobian scales the gradient by roughly the inverse of the activation magnitude. Stacked deep, these repeated rescalings attenuate the gradient reaching layer 1, so post-norm transformers are notoriously hard to train past a couple dozen layers without careful warmup and small learning rates — the identity path is not actually clean, because a norm sits on it at every block.
In pre-norm, the computation is x + Sublayer(LayerNorm(x)): the normalization is inside the branch, applied only to the input of the sublayer, and the residual add is unnormalized. Now the identity path from the very first layer to the output is a pure sum of terms with nothing rescaling it — the gradient flows back through the additive skip essentially unattenuated, mathematically close to multiplying by one at each hop. This is why pre-norm networks train stably at depths where post-norm diverges, why they need far gentler (or no) warmup, and why they became the default the moment models pushed past a few dozen layers. The cost, and the reason the story is not entirely one-sided, is representational, which the next section makes precise.
One way to build intuition is to think about what each design does to the variance of the signal as it travels the network. Post-norm renormalizes after every block, so the forward activations have roughly unit variance everywhere — beautifully conditioned for the forward pass, but that renormalization is a multiplicative operation sitting on the shortest path from output to input, and multiplicative operations stacked deep are exactly what erodes a gradient. Pre-norm never touches the residual sum, so its variance grows roughly linearly with depth as each block adds its contribution to the running total; the forward signal is less tidy, but the backward path down the residual stream is a clean chain of additions whose derivative is one. The whole pre-norm-versus-post-norm debate can be read as a choice about where to spend your variance budget: post-norm spends it keeping the forward pass pristine and pays on the backward pass, while pre-norm lets the forward pass drift and spends nothing on the backward pass — and since trainability at depth is gated by the backward pass, pre-norm wins wherever depth is the objective.
The architecture: every piece explained
Top row: the two placements side by side. Post-norm — the original design — computes each sublayer as x = LayerNorm(x + Sublayer(x)). Every block's output is freshly normalized, so activations never grow across depth; the network is well-conditioned at the forward pass, but the normalization sits on the residual path and taxes the backward pass. Pre-norm — the modern default — computes x = x + Sublayer(LayerNorm(x)). The sublayer sees a normalized input (stable, well-scaled), but its output is added back to an un-normalized residual, so the activations on the residual stream grow as you go deeper: each layer adds its contribution and nothing rescales the running sum.
Middle row: the consequences. The residual stream is clean in pre-norm (ungated identity) and gated by a norm in post-norm — the single most important difference. Gradient flow is therefore near-unattenuated in pre-norm and progressively attenuated in post-norm. This directly sets the warmup need: post-norm requires a long, careful learning-rate warmup to survive the early-training instability when gradients are ill-behaved, whereas pre-norm tolerates much shorter warmup because its gradients are well-conditioned from the start. And because pre-norm's residual stream grows unboundedly and is never normalized inside the stack, pre-norm requires a final LayerNorm after the last block to rescale activations before the output projection — post-norm does not, since its last operation is already a norm.
Bottom rows: the trade-off and the scaling story. Depth scaling is where pre-norm wins decisively: it is what makes 100+ layer transformers trainable, and every very deep model uses it or a stabilized variant. The stability vs representation tension is the honest counterpoint: because pre-norm's residual grows with depth, the later layers' sublayers contribute proportionally less to the ever-larger running sum — an effect that can make very deep pre-norm networks behave, in their upper layers, a little like shallower ones, sometimes costing a sliver of final quality versus a well-tuned post-norm network of the same depth. This is why refinements exist — sandwich norm (norm before and after the sublayer), DeepNorm and other residual scalings that make post-norm trainable deep, and QK-norm for attention-logit stability. The ops strip is the practitioner's dashboard: watching for loss spikes, tracking gradient norms and activation growth through depth, and setting initialization scaling to keep the residual stream in range.
End-to-end flow
Walk one training step through both variants of a 48-layer decoder to see where they diverge. A batch of tokens enters as embeddings, forming the initial residual stream of shape (batch, seq, d_model). Forward pass, pre-norm: block 1 normalizes its input, runs attention, and adds the result back — the residual sum is now embeddings plus a modest attention contribution. Block 2 does the same, and the running sum grows a little. By block 48, the residual stream has accumulated 48 sublayer contributions and has a substantially larger norm than it started with; the final LayerNorm rescales it back to a sane range before the unembedding projection produces logits. Nothing rescaled the stream in between, which is exactly why it grew — and exactly why the gradient will flow back cleanly.
Forward pass, post-norm: block 1 runs attention, adds to the input, and then normalizes the sum — so the residual entering block 2 is already unit-scaled, and it stays that way at every block. The forward activations are beautifully conditioned end to end, no growth, no final norm needed. The difference only bites on the backward pass. The loss produces a gradient at the logits; it flows back through the unembedding. In pre-norm, it enters the residual stream and travels back through 48 additive skips — at each block it splits, part going through the (normalized) sublayer and part continuing straight down the identity path unscaled. The gradient reaching block 1's parameters is essentially as strong as the gradient at block 48: deep learning, working as intended. In post-norm, that same gradient must pass through a LayerNorm at every one of the 48 blocks on its way down the residual path, each one rescaling it by roughly the inverse activation magnitude; by block 1 it has been attenuated many times over, so the early layers receive a much weaker signal and learn slowly — or, early in training before warmup has tamed the learning rate, the interaction of large ill-conditioned gradients with these norms produces a loss spike and the run diverges.
This is precisely why the practitioner reaches for pre-norm at depth: the same 48 layers that train smoothly with a short warmup and a standard learning rate under pre-norm demand a long warmup, a smaller learning rate, and careful initialization under post-norm — and still get harder to train as you add layers. And it is why, when a team does want post-norm's cleaner representation at depth, they don't use vanilla post-norm but a stabilized variant like DeepNorm, which scales the residual branch by a depth-dependent constant so the gradient survives — the refinement that reconciles post-norm's forward conditioning with pre-norm's trainability.
The practical upshot for anyone bringing up a new model is to treat norm placement as a decision made before the first training run, not a knob to tweak after it diverges. The placement dictates the warmup schedule, the peak learning rate, the initialization scheme, and whether a final normalization is required — a coupled bundle of choices, not an isolated one. Changing the placement mid-project means re-tuning all of them, which is why nearly every team simply adopts pre-norm with a final LayerNorm as the safe default and only departs from it, with a stabilized variant and its prescribed recipe, when they have a concrete reason to chase post-norm's forward conditioning at extreme depth.