Why architecture matters here

Druid fails on wrong rollups + missed capacity. Architecture matters because ingest + storage + query all balance.

Advertisement

The architecture: every piece explained

The top strip is ingest to serving. Ingest. Segments. Historicals. Brokers.

The middle row is optimization. Coordinator. Rollups. Bitmap indexes. Deep storage.

The lower rows are practice. Query cache. Use cases. Ops — sizing + retention + concurrency.

Druid — segments + brokers + historicals + real-time ingest + rollupssub-second OLAP on streaming + batchIngeststreaming + batchSegmentscolumnar chunksHistoricalshold segmentsBrokersquery fan-outCoordinatorload balanceRollupspre-aggregate at ingestBitmap indexesfast filtersDeep storageS3 / HDFS backupQuery cacheper-brokerUse casesclickstream / metrics / observabilityOps — sizing + retention + concurrencydistributeaggregatefilterpersisthitapplyapplyoperateoperate
Druid tiered architecture with brokers + historicals.
Advertisement

End-to-end flow

End-to-end: Kafka stream ingests. Rollups reduce to per-minute aggregations. Segments built + pushed to historicals. Broker fans queries. Bitmap indexes accelerate WHERE. Query cache serves repeats.