HBase vs DynamoDB, in depth: key models, ordering, atomicity, capacity and hot keys, translating a row-key design, and when to choose which

Two answers to the same question

In HBase, a table is a sorted map from row key to a set of column families, each holding columns, each holding timestamped versions. The full key space is cut into contiguous ranges called regions, and each region is served by exactly one RegionServer at a time. A client finds the region for a row by consulting the hbase:meta table, caches the location, and talks directly to that RegionServer. Because regions are ranges of a sorted space, a scan from key a to key m visits keys in order across however many regions that range covers.

In DynamoDB, every item has a partition key and optionally a sort key. The service hashes the partition key to choose an internal partition, so two partition key values that differ by one character can land anywhere. Within a partition key value, items are stored in sort-key order, and that collection can be queried by range. There is no operation that returns items across partition keys in key order; a Scan visits everything in an unspecified order and charges you for all of it.

Feature by feature

Atomicity and consistency

HBase guarantees that a mutation to a single row is atomic, however many columns it touches, and offers checkAndMutate for compare-and-set on one row. Reads of a row after a successful write see that write, because one RegionServer owns the row. There are no multi-row transactions in core HBase; projects that need them add a layer such as Apache Phoenix with a transaction manager, or design so that everything that must change together lives in one row.

DynamoDB guarantees atomicity for one item and offers condition expressions on writes, which play the role of checkAndMutate. Reads are eventually consistent by default and strongly consistent on request for the base table and local secondary indexes, at twice the read-capacity cost. Global secondary indexes are always eventually consistent. Multi-item ACID transactions are available through TransactWriteItems and TransactGetItems, capped at a fixed number of items and 4 MB in aggregate; the item cap has been raised in the past, so check the current quota rather than designing to a remembered number.

Across regions the picture changes again. HBase replication is asynchronous and ships WAL edits to peer clusters, so a failover can lose the most recent writes and concurrent writes in two clusters resolve by timestamp. DynamoDB global tables historically offered multi-active replication with last-writer-wins; since June 2025 AWS also offers global tables with multi-Region strong consistency, where a write is replicated to at least one other region before it is acknowledged. That mode costs write latency and is only available in a subset of regions, so it is a deliberate choice rather than a default. The details of the original model are in DynamoDB global tables.

Throughput, capacity and hot keys

Both systems have a hot-key problem, and they fail in different ways. In HBase, a monotonically increasing row key such as a timestamp sends every write to the last region, so one RegionServer saturates while the rest idle. The standard cures, salting, hashing a prefix or reversing the key, are covered in HBase hotspotting. The ceiling on a single hot region is whatever that server can do, which is often tens of thousands of small writes per second, and when it is exceeded latency climbs and the MemStore blocks writes until flushes catch up.

In DynamoDB, the documented ceiling is per partition: 3,000 read capacity units and 1,000 write capacity units per second. One write capacity unit is one write per second of an item up to 1 KB; one read capacity unit is one strongly consistent read per second of up to 4 KB, or two eventually consistent ones. Adaptive capacity isolates hot items and can split an item collection by sort key, but one item never exceeds a partition's maximum, and a collection is not split when the table has a local secondary index or traffic follows a monotonically increasing sort key, which is exactly the timestamp pattern. Excess load returns throttling errors the SDK retries with backoff.

The practical consequence: a key design that was merely slow in HBase can be a hard wall in DynamoDB. A counter row updated by every request, a "latest" pointer, or a tenant identifier used alone as the partition key for a very large tenant all need write sharding, where you append a suffix such as #0 to #9 and fan reads out across the shards.

The same operation in both

A small example makes the API differences concrete. The task: record a sensor reading, then read the last hour of readings for one device. In HBase the row key is the device identifier followed by a reversed timestamp, so the newest reading sorts first.

// HBase Java client
byte[] row = Bytes.add(Bytes.toBytes(deviceId), Bytes.toBytes(Long.MAX_VALUE - tsMillis));
Put put = new Put(row);
put.addColumn(CF, Bytes.toBytes("temp"), Bytes.toBytes(temperature));
table.put(put);

// last hour: scan from "now" to "now - 1h" for one device prefix
byte[] start = Bytes.add(Bytes.toBytes(deviceId), Bytes.toBytes(Long.MAX_VALUE - nowMillis));
byte[] stop  = Bytes.add(Bytes.toBytes(deviceId), Bytes.toBytes(Long.MAX_VALUE - (nowMillis - 3_600_000L)));
Scan scan = new Scan().withStartRow(start).withStopRow(stop).addFamily(CF);
try (ResultScanner rs = table.getScanner(scan)) {
    for (Result r : rs) { /* newest first */ }
}

In DynamoDB the device identifier becomes the partition key and the timestamp the sort key. Ordering within the device is preserved, so the query is a key-condition range, and ScanIndexForward=False returns newest first.

# DynamoDB with boto3
import boto3
from boto3.dynamodb.conditions import Key

table = boto3.resource("dynamodb").Table("readings")
table.put_item(Item={"device_id": device_id, "ts": ts_millis, "temp": temperature})

resp = table.query(
    KeyConditionExpression=Key("device_id").eq(device_id)
        & Key("ts").between(now_millis - 3_600_000, now_millis),
    ScanIndexForward=False,
)
items = resp["Items"]
while "LastEvaluatedKey" in resp:          # a Query page holds at most 1 MB
    resp = table.query(..., ExclusiveStartKey=resp["LastEvaluatedKey"])
    items += resp["Items"]

Two differences hide in this example. First, the HBase design spreads load only if device identifiers are well distributed; DynamoDB spreads by hash regardless. Second, the HBase version can also answer "all devices whose identifier starts with plant7-" with a prefix scan; DynamoDB cannot without a secondary index whose partition key is the plant.

Worked example: translating a row-key design

Consider a real HBase table for an e-commerce order history. The row key is customerId|reverseTs|orderId. The table answers three questions: a customer's recent orders (prefix scan on customerId|), one order by identifier (a second table keyed by orderId holding the main key), and a nightly export of everything changed yesterday (a full scan with a timestamp filter, run by a Spark job).

Translating it means listing access patterns first, then choosing keys, which is the same discipline single-table design teaches:

Notice what disappeared. The second lookup table becomes a global secondary index the service maintains, which removes a whole class of "index row written, main row failed" bugs, but the index is eventually consistent, so a read immediately after an order is placed may miss it. The nightly scan is the expensive one: a full-table Scan in DynamoDB consumes read capacity for every item, so the idiomatic replacement is change data capture through Streams, or the managed export to S3 for analytics, rather than a nightly scan.

A migration sketch

Moving data from HBase to DynamoDB is a backfill plus a catch-up, the same shape as any live migration. Start dual-writing new changes, or capture them through HBase replication to a custom endpoint, then backfill from a snapshot, then verify and cut reads over.

# Backfill sketch: read an HBase snapshot offline, transform, write in batches
for region_split in snapshot_input_splits("orders_snap"):        # e.g. TableSnapshotInputFormat in Spark
    batch = []
    for row in read_split(region_split):
        item = to_item(row)                 # CUST#id / ORDER#ts#id, drop old versions
        if item_size(item) > 400 * 1024:
            raise ValueError(f"oversize item {row.key}")
        batch.append(item)
        if len(batch) == 25:                # BatchWriteItem accepts at most 25 puts or deletes
            write_with_retry(batch)         # resend UnprocessedItems with backoff
            batch = []
    write_with_retry(batch)

Reading the snapshot rather than the live table keeps the backfill off the RegionServers. On the DynamoDB side, the backfill rate is limited by write capacity and by hot partitions: if the source is sorted by customer and you write it in that order, consecutive writes hit the same partition. Shuffle the input before writing. For very large new tables, importing from files in S3 into a new table avoids consuming write capacity at all. The verification and reversible-cutover steps in HBase migration apply unchanged: compare counts and hashes per key range before you move reads.

Failure modes

• Assuming global order survives. Any code that scans a key range across entities silently becomes a full Scan or stops working. Inventory every scan before migrating.
• One hot partition key. A global counter item, or a time-ordered collection, throttles at the per-partition rate. Shard the key and aggregate on read.
• Reading a GSI as if it were consistent. Read-your-write flows break intermittently. Read the base table for anything that must reflect the write just made.
• Version history lost. HBase applications sometimes read older cell versions for audit. DynamoDB keeps one version; model history as separate items or stream changes to an archive.
• Cost surprise. A workload of large rows read in full, or analytics scans, can cost far more per month in DynamoDB than a cluster. Estimate request units from real traffic, not averages.

Choosing

Choose HBase when you already run Hadoop and need global ordered scans, very wide rows, cell versions, server-side processing next to the data, or offline bulk loads of terabytes at a time, and you have people who can operate it. Choose DynamoDB when your access patterns are known and keyed by entity, you want no infrastructure to run, you need predictable single-digit-millisecond reads at any scale, or you need managed multi-region replication.

What to do next

1. Write down every access pattern your HBase application uses, marking each as point get, prefix or range scan within an entity, or scan across entities.
2. For each cross-entity scan, decide whether it becomes a secondary index, a stream-fed pipeline, or an export to analytics storage.
3. Draft partition and sort keys, then estimate peak writes per item and per time-ordered collection against 1,000 write units per second.
4. Measure your largest rows and plan how to split anything near 400 KB.
5. Estimate monthly cost from real request counts and item sizes, not averages, and compare it with the full cost of the cluster including staff time.
6. Prototype the two or three hottest access patterns in DynamoDB with production-shaped data before committing.
7. If you migrate, backfill from a snapshot, catch up from a change feed, verify by key range, and keep the HBase path reversible until reads have run cleanly for a week.