Core concept

.foreachBatch { (df, id) => df.write.mode("append").format("delta").save(...) }
Advertisement

How it works

Batch ID for idempotency. Function called per batch.

Advertisement

Trade-offs + gotchas

Complex writes (multiple sinks, MERGE INTO).