Why architecture matters here

Watermark bugs are hard to spot. Data appears to be counted correctly until a late-arriving event 20 minutes past the window boundary is silently dropped. The architecture matters because event-time semantics require deliberate delay + output mode choices.

With the pieces mapped, you can define correctness for your streaming pipeline explicitly.

Advertisement

The architecture: every piece explained

The top strip is the semantics. Source reads events. Event time column annotates each row. Watermark is (max event time seen) − allowed delay; state older than watermark is safe to close. State store holds windowed aggregations.

The middle row is control. Trigger defines the processing-time cadence (continuous, once, or interval). Output mode — append (only closed windows), update (any changed row), complete (whole state) — must match sink. Late data policy decides whether to drop or route past-watermark data to a side output. Checkpoint to durable storage enables restart.

The lower rows are sinks + ops. Sink: Delta, Kafka, or files. Observability tracks batch duration + lag. Ops covers schema evolution, state pruning, and backfill.

Structured streaming watermarks — event time + late data + trigger + outputcorrectness under late arrivalSourcekafka / filesEvent time columnassigned per rowWatermarkmax seen − delayState storewindowed aggregationsTriggerprocessing time cadenceOutput modeappend / update / completeLate data policydrop / markCheckpointrecover on failureSinkdelta / kafka / filesObservabilitybatch duration + lagOps — schema evolution + state pruning + backfilltriggeroutputpolicycheckpointwritewatchwatchoperateoperate
Structured streaming watermark pipeline.
Advertisement

End-to-end flow

End-to-end: job aggregates page views per 5-minute window with watermark delay 10 minutes. Trigger interval 1 minute. State grows during the last 10 minutes; watermark closes older windows and outputs final counts in append mode. A late event 8 minutes past its window still gets counted (inside the delay). A late event 12 minutes past is dropped (or routed via late-data sink). Checkpoints allow restart from last processed offsets. Observability shows batch duration steady + lag under 1 minute.