Core concept
@pandas_udf("double")
def my_udf(v: pd.Series) -> pd.Series:
return v * 2Advertisement
How it works
Requires PyArrow. Batch size tuning.
Advertisement
Trade-offs + gotchas
Python UDFs on PySpark.
@pandas_udf("double")
def my_udf(v: pd.Series) -> pd.Series:
return v * 2Requires PyArrow. Batch size tuning.
Python UDFs on PySpark.