Why architecture matters here
To see why the architecture matters, count the work a single row does in the Volcano model. A query that scans a column, filters it, and sums it touches three operator objects. For every row, the aggregate calls next() on the filter, which calls next() on the scan, which decodes a value, returns it up the chain through two virtual dispatches, gets tested against the predicate, and returns again. That is several polymorphic method calls, likely a boxed Long allocation, and a load-store round trip through the heap — per row, for billions of rows. The actual arithmetic, the addition, is a single instruction buried under a mountain of dispatch overhead. Modern CPUs are astonishingly fast at the arithmetic and astonishingly sensitive to the overhead: mispredicted branches and instruction-cache misses from polymorphic dispatch can cost more than the useful work by an order of magnitude.
Whole-stage codegen matters because it collapses that entire chain into a loop that a hand-writing expert would produce. The generated function reads the column value into a register, evaluates the filter predicate as an inline branch, and accumulates the sum in another register — no method calls, no boxing, no heap traffic, and a branch pattern the predictor learns immediately. This is the difference between an engine that is bounded by dispatch overhead and one that is bounded by the memory bandwidth of reading the data, which is the theoretical floor for a scan-heavy query.
The second reason the architecture is load-bearing is that it changes what the JVM's JIT compiler can do. The JIT is excellent at optimizing straight-line, monomorphic code and poor at optimizing megamorphic virtual call sites — exactly the pattern the Volcano model produces, where one next() call site is invoked with dozens of different operator implementations across a workload. By generating a single concrete class per stage with no polymorphism on the hot path, codegen hands the JIT code it can inline aggressively, keep in registers, and vectorize where possible. The engine is, in effect, cooperating with the JIT instead of defeating it.
There is a third, strategic reason: codegen is what lets Spark exploit the columnar, off-heap memory layout that the rest of Tungsten provides. Cache-friendly binary rows and columnar batches are only fast if the code reading them is also fast; a Volcano chain would squander a good memory layout on dispatch overhead. Codegen closes the loop — efficient data format plus generated code that reads it in tight loops — and that pairing is why Spark's SQL engine competes with purpose-built analytical databases rather than merely being a distributed batch runner. Understanding codegen is therefore understanding where the bulk of Spark SQL's single-core performance actually comes from.
The architecture: every piece explained
Top row: from plan to source. After the Catalyst optimizer produces a physical plan — a tree of operators like Scan, Filter, Project, and HashAggregate — a rule called CollapseCodegenStages walks the tree and groups maximal runs of adjacent operators that implement the CodegenSupport trait into a single WholeStageCodegenExec node. Each supporting operator knows how to emit two kinds of Java: a doProduce fragment that drives the loop (usually originating at the scan) and a doConsume fragment that processes one row and hands it to its parent. Chaining these produce/consume fragments yields the fused Java source: one method whose body is the scan's loop with the filter, projection, and aggregate logic inlined directly inside it, operating on values in local variables rather than on iterator objects.
Middle row: from source to running code. That generated source string is handed to Janino, a lightweight runtime Java compiler, which compiles it into a generated class — a subclass of BufferedRowIterator whose processNext() contains the fused loop. Spark caches these compiled classes keyed on the generated source so identical stages across tasks and partitions compile once. When the stage runs, the JVM loads the class and the JIT compiles the loop to machine code, keeping intermediate values in registers, eliminating the virtual calls that the Volcano model would have incurred, and applying loop optimizations the polymorphic path could never receive.
Bottom rows: the boundaries. Codegen fuses within a stage but a stage boundary — any shuffle or exchange, such as the map side and reduce side of a join or aggregation — breaks fusion, because data must be repartitioned across the network between them; each side becomes its own whole-stage codegen unit. Not every operator supports codegen: some sources, certain complex expressions, and operators that predate or opt out of the mechanism take the fallback path, running in the classic row-at-a-time mode and appearing in the plan without a codegen id. The ops strip names your instruments: df.explain("codegen") prints the generated source, the *(n) markers in a physical plan show which operators fused into whole-stage id n, and the 64KB JVM method-size limit is the guard rail every large generated loop must respect.
The produce/consume protocol deserves emphasis because it is what makes fusion compositional rather than a special case per query shape. Rather than each operator owning a loop and pulling from a child, the leaf operator that can drive iteration (a scan reading a columnar batch) owns the single loop via doProduce, and every operator above it contributes a doConsume snippet that is textually spliced into that loop body in tree order. Because the snippets share generated local variables — the current row's columns live in named Java locals threaded through the chain — the boundary between operators disappears entirely in the emitted code. That is the core trick: the operator tree is a compile-time construct that produces a flat loop, and nothing of the tree's indirection survives into the running function.
End-to-end flow
Trace a concrete query: spark.table("sales").filter($"amount" > 100).groupBy($"region").sum("amount"). Catalyst optimizes it and produces a physical plan whose scan reads the amount and region columns, a filter tests amount > 100, and a HashAggregate accumulates the per-region sum. Because the aggregation requires a shuffle to bring all rows of a region together, the plan splits at an Exchange into two stages: a map-side partial aggregate and a reduce-side final aggregate.
Codegen assembly. CollapseCodegenStages examines the map stage and finds Scan, Filter, and partial HashAggregate all support codegen, so it wraps them in one WholeStageCodegenExec. The scan's doProduce emits a loop over the columnar batch; the filter's doConsume emits if (amount > 100) inline; the partial aggregate's doConsume emits the hash-map update that accumulates the running sum by region. The three fragments become one method. Janino compiles it; the class is cached and reused across every partition of the map stage.
Execution. Each task runs the generated loop over its partition's rows. A value is decoded into a register, compared against 100 with an inline branch, and — if it passes — folded into the region's running sum in an off-heap hash map, all without a single virtual call or heap allocation per row. This is the stage that used to be dominated by iterator dispatch; now it runs at close to memory-scan speed. The partial aggregates are written to shuffle files.
Across the boundary. The Exchange repartitions by region; the reduce stage is a second whole-stage codegen unit whose generated loop reads the shuffled partial sums and combines them into final per-region totals. Two generated classes, one per stage, joined by a shuffle. When you call df.explain("codegen"), you can read both generated methods; when you view the physical plan, the fused operators carry a *(1) and *(2) prefix marking their whole-stage ids, while the Exchange between them carries none because a shuffle is never inside a codegen stage. If, instead, the filter had used a user-defined function Spark couldn't codegen, that operator would drop to the fallback path, the fusion would break around it, and the plan would show the UDF operator without a star — an immediate visual signal that a row-at-a-time island had opened inside an otherwise-fused stage, usually the first thing to investigate when a query is slower than its shape suggests.
It is worth quantifying what the fusion actually bought on this query, because the number is what justifies caring about the plan at all. In the Volcano path, each of the map stage's rows crossed three operator boundaries — scan to filter to partial aggregate — with a virtual call and, for the sum, a boxed Long at every hop, so the per-row cost was dominated by dispatch and heap traffic rather than by the comparison and the addition that are the only real work. In the fused path, the same row is decoded into a register, compared inline, and folded into the aggregate's hash slot without a single method call or allocation, so the loop runs close to the rate at which the CPU can stream the column through cache. On a billion-row scan that is the difference between a stage bounded by JVM dispatch overhead and one bounded by memory bandwidth — frequently an order of magnitude — and it is realized purely by CollapseCodegenStages recognizing that the three operators could share one loop, with no change to the query the user wrote.