Core concept

val df = spark.read.parquet("path")
df.write.parquet("out")
Advertisement

How it works

Row group size + compression matter.

Advertisement

Trade-offs + gotchas

Default for Spark analytical.