Core concept
val myUDF = udf((s: String) => s.toUpperCase)
df.select(myUDF($"name"))Advertisement
How it works
Serialization overhead per row. Prefer built-ins or Pandas UDFs.
Advertisement
Trade-offs + gotchas
Custom logic not in built-ins.