Why it matters

Managed streaming eliminates ops. Understanding Kinesis helps AWS-native architectures.

Advertisement

The architecture

Streams: named event stream. Partitioned by shards.

Shards: unit of throughput. 1 MB/s in, 2 MB/s out per shard.

Kinesis modelProducerPutRecord(s)Shardsthroughput unitsConsumerGetRecords or Enhanced Fan-outEnhanced Fan-out gives each consumer dedicated throughput; standard has shared
Kinesis Data Streams.
Advertisement

How it works end to end

Enhanced fan-out: dedicated throughput per consumer. Vs shared throughput with GetRecords.

Retention: 24 hours default, up to 1 year.

Kinesis Firehose: managed delivery to S3, Redshift, Elasticsearch. No consumer to write.

Client library: KCL handles checkpointing, rebalancing similar to Kafka consumers.