Why it matters

Watermarks essential for event-time streaming.

Advertisement

The architecture

Watermark = max event time - allowed lateness.

Data before watermark: dropped.

Watermark mechanicsEvent time trackingmax seen - latenessWatermark advancesas events arriveLate datadropped past watermarkTrade-off: shorter lateness = smaller state, more late data drops
Watermark trade-off.
Advertisement

How it works end to end

Allowed lateness: how late data can be.

State cleanup: past watermark can be dropped.

Late data: sink-specific handling (drop, alert).