Why architecture matters here
Custom functions matter because the alternative to a UDF is often a worse architecture: pre-computing the custom logic in a separate pipeline (staleness, extra system), exporting data to compute externally (data movement, no pushdown), or contorting SQL into an unmaintainable expression. A UDF keeps the custom logic in the query, running distributed across the cluster next to the data, composable with the rest of SQL, and optimizable (predicate pushdown around it, parallelism through it). For genuinely custom computation — domain algorithms, proprietary formats, specialized aggregations — a UDF is the right place, and the ability to extend SQL rather than escape it is a real architectural advantage.
The reason the vectorization cost dominates the UDF story is that it's silent and total. A query with a non-vectorized UDF in its projection doesn't error or warn — it just runs the affected operators row-at-a-time (or through the VectorUDFAdaptor's unpack-repack), 5× slower for identical results, and the slowdown appears in production as unexplained latency. One convenience UDF added to a hot dashboard query can quintuple its cost forever, and the investigation ends at EXPLAIN VECTORIZATION showing the fallback. This makes UDF vectorization not a nice-to-have but a first-class design concern for any function on a performance-sensitive path: either implement it as a VectorizedExpression (restoring the fast path) or accept and measure the adaptor overhead, never ship a silent row-mode UDF on tier-1 queries.
UDAFs carry a subtler but equally important contract: partial aggregation correctness. A distributed aggregate must compute partial results on each mapper (map-side partial aggregation) and merge them on reducers — so a UDAF must implement not just 'aggregate these rows' but 'produce a mergeable partial state' and 'merge two partial states'. Get this wrong (a UDAF that only works if it sees all rows) and it's either incorrect at scale or forces all data through one reducer (killing parallelism). The partial-aggregation model is the same insight as combiners in MapReduce and partial aggregates in every distributed engine, and a UDAF that respects it scales while one that doesn't becomes a bottleneck — the difference between a custom aggregate that works on billions of rows and one that OOMs.
The architecture: every piece explained
Top row: the function types. A UDF (legacy, reflection-based) is the simplest — a class with an evaluate method taking and returning primitives; fine for simple scalar functions but limited (no complex types, less control). GenericUDF is the modern API: it uses ObjectInspectors to handle any type (primitives, structs, lists, maps, variable arguments), with explicit initialize (type-check arguments, declare return type via an ObjectInspector) and evaluate (compute per row) methods — the right choice for anything non-trivial. A UDAF (GenericUDAFResolver + evaluator) implements aggregation with the partial-aggregation lifecycle: iterate (add a row to partial state), terminatePartial (emit partial state for merging), merge (combine partial states), terminate (final result) — the four methods that make it distributed-correct. A UDTF generates rows: one input row produces zero or more output rows (via a forward callback), used with LATERAL VIEW.
Middle row: performance and types. Vectorized UDF is the fast path: implementing VectorizedExpression processes a whole column batch at once (reading input ColumnVectors, writing an output ColumnVector) — keeping the query on the vectorized path instead of falling to row mode; the difference between a UDF that costs 5× and one that costs nothing extra. Partial aggregation is the UDAF's scalability: map-side partial aggregates (each mapper aggregates its rows into partial state) reduce shuffle volume and reduce-side merging combines them — the model that lets aggregates scale. ObjectInspectors are Hive's type system at runtime: they describe the structure of values (a StructObjectInspector for a struct, ListObjectInspector for a list) so generic functions introspect and handle arbitrary types without compile-time knowledge of them. Registration: functions register as temporary (session-scoped, CREATE TEMPORARY FUNCTION) or permanent (metastore-registered, available to all sessions, tied to a JAR location).
Bottom rows: the operational reality. Distribution and dependencies: the UDF's JAR (and its dependencies) must be available on every node executing the query — via ADD JAR, permanent function JAR locations (HDFS/object store), or the cluster's classpath — and dependency conflicts (the UDF's libraries versus Hive's) are a classic source of ClassNotFound and version-mismatch failures. Determinism and statefulness are correctness contracts: the optimizer assumes UDFs are deterministic (same input → same output) and stateless unless annotated otherwise (@UDFType(deterministic = false) for things like rand() or current_timestamp()) — a UDF that's secretly non-deterministic but not annotated can be incorrectly optimized (cached, reordered) producing wrong results. The ops strip: vectorization impact (the dominant concern — measure and implement vectorized where it matters), JAR management (distribution, versioning, dependency isolation), and testing (unit-test the function logic, integration-test in Hive, verify vectorization).
End-to-end flow
Build a custom aggregate correctly and watch the partial-aggregation model matter. The need: a custom weighted_median(value, weight) aggregate not in Hive's built-ins. The naive implementation collects all (value, weight) pairs and computes the median at the end — which works on small data and dies on large: it forces every row through one reducer (no partial aggregation) and OOMs on billions of rows. The correct GenericUDAF implements the lifecycle: iterate adds pairs to a bounded partial state (a compressed representation — say a t-digest-like sketch), terminatePartial emits the sketch, merge combines sketches, terminate computes the median from the merged sketch. Now it scales: each mapper builds a small sketch (map-side partial aggregation), reducers merge sketches (tiny shuffle), and the median comes from the merged sketch — billions of rows, bounded memory, full parallelism. The partial-aggregation model turned an impossible aggregate into a scalable one.
The vectorization vignette shows the UDF performance trap and its fix. A normalize_score(raw, min, max) UDF is added to a hot dashboard query. Implemented as a plain GenericUDF, it forces the query's projection to row mode — the dashboard slows from 3s to 15s, and EXPLAIN VECTORIZATION shows the fallback with 'UDF not vectorized'. The team implements it as a VectorizedExpression: reading the raw/min/max ColumnVectors and writing the normalized output ColumnVector in a tight loop — restoring the vectorized path and the 3s latency. The cost was a day of implementing the vectorized version; the alternative was a permanent 5× tax on a tier-1 query. The team's policy crystallizes: UDFs on performance-sensitive paths get vectorized implementations, and EXPLAIN VECTORIZATION is checked in review for any query using custom functions.
The operational vignettes complete it. A UDF's JAR pulls in a JSON library that conflicts with Hive's version — ClassNotFoundError in production, a classic dependency clash; the fix is shading the UDF's dependencies (relocating them to avoid the conflict) and testing the JAR in an environment matching production's classpath. A different UDF, enrich_from_cache(id), secretly calls an external service (non-deterministic, stateful) but isn't annotated — the optimizer, assuming determinism, caches results across rows that should differ, producing wrong output; the fix is the @UDFType(deterministic = false) annotation (and, better, questioning whether a UDF should call external services at all — usually a design smell). The consolidated discipline the team documents: choose the right function type (UDF/UDAF/UDTF), implement UDAFs with correct partial aggregation, vectorize functions on hot paths, manage JARs and dependencies (shade conflicts), annotate determinism honestly, and test both logic and vectorization — because a UDF is custom code running in the query engine's hot path, and it must respect the engine's contracts or it silently breaks correctness or performance.