Why architecture matters here
A RegionServer is a shared machine with finite handlers, memory, and disk bandwidth, serving requests for many regions on behalf of many tenants. Its default scheduling is essentially fair-queued but unbounded: any client can submit as fast as it can, and the RegionServer will try to serve everyone. That works until one tenant's traffic dwarfs the rest — an analytics job launching thousands of concurrent full-table scans, or a batch loader firing multi-megabyte puts in a tight loop. Those requests consume handler threads, evict hot data from the block cache, and generate flush and compaction pressure, and the damage lands on every tenant sharing the server, not just the one causing it.
The reason this needs an in-server architecture rather than client-side politeness is trust and enforcement. You cannot rely on every client to rate-limit itself: clients are written by different teams, run at different scales, and misbehave under load or bugs precisely when restraint matters most. Enforcement has to sit at the resource being protected — the RegionServer — where it can observe actual consumption and say no. And it has to be metered, not binary: the goal is not to block a tenant but to hold them to a sustainable share, letting bursts through when there is slack and shedding excess when there is not.
Throttle quotas provide exactly that: a per-principal contract that says 'you may sustain N requests per second and B bytes per second, with a burst allowance, and beyond that your requests will be rejected with a retriable exception.' This converts an unbounded shared resource into a set of enforced, isolated shares. It also gives operators a knob to encode business priority — a latency-sensitive serving path gets a generous quota, a best-effort batch job gets a modest one — so that when the cluster is contended, the important traffic wins by policy rather than by luck. The same principle underlies rate limiting in API gateways and databases generally; HBase's contribution is doing it at the granularity of user, table, and namespace inside a region-serving process.
The metering-not-blocking distinction is what separates a quota from a circuit breaker, and it is central to why throttling is usable in production. A circuit breaker is binary and reactive: it trips open after damage is detected and rejects everything until it closes. A token-bucket quota is continuous and proactive: it admits traffic up to a sustainable rate at all times, smoothly rejecting only the excess, so a well-behaved client operating just under its limit never sees a single rejection while a client that spikes sees exactly the overflow shed. That gradient is what makes quotas safe to leave permanently enabled — they are not an emergency measure that hurts everyone when it fires, but a steady-state fairness mechanism that most clients never notice. It also means the right quota is one that a tenant's normal traffic fits comfortably under, with the ceiling positioned to catch pathological bursts rather than to shape ordinary load.
The architecture: every piece explained
Quota definitions are the source of truth and live in the hbase:quota system table, managed by the Master and set through the shell (set_quota TYPE => THROTTLE, USER => 'u1', LIMIT => '100req/sec') or the admin API. A throttle quota specifies a type (request number, request size, read number, write number, read size, write size) and a limit over a time unit (per second, minute, hour, or day). Its scope can be a user, a table, a namespace, a user-on-a-table, or a user-in-a-namespace, and HBase evaluates all applicable quotas for a request, so a user quota and a table quota both constrain the same operation.
Each RegionServer maintains a quota cache that periodically refreshes definitions and current usage from the quota table (the refresh period is configurable, on the order of minutes). For each quota, the cache holds a token bucket: a limiter with a refill rate equal to the quota limit and a capacity that permits short bursts. A request must acquire enough tokens to cover its cost; the bucket refills continuously at the configured rate, so sustained traffic is capped at the limit while brief spikes draw down the accumulated burst allowance. This is the standard token-bucket shape, applied independently to request-count and byte-throughput dimensions.
The subtle part is cost estimation. For a put, the size is known up front. For a scan, the RegionServer cannot know how many bytes it will return before executing it, so throttling uses an estimate-then-consume protocol: it estimates the operation's cost, checks that the buckets have enough tokens, reserves the estimate, executes the operation, then reconciles the reservation against the actual bytes and rows processed — refunding an over-estimate or charging the difference. This keeps large scans honestly accounted without requiring the server to run them before deciding whether they are allowed.
Enforcement is wired into the RPC path ahead of execution and cooperates with the RPC scheduler. When a request cannot acquire its tokens, the RegionServer throws an RpcThrottlingException carrying a waitInterval — how long the client should back off before retrying — so throttling is a retriable, self-pacing signal rather than a hard failure. Separately, space quotas live in the same system table and, when a table or namespace exceeds its size limit, apply a violation policy (NO_INSERTS, NO_WRITES, NO_WRITES_COMPACTIONS, or DISABLE) that the RegionServers enforce until usage returns below the limit.
A structural consequence worth understanding is that throttle enforcement is per RegionServer, not global. Each server independently maintains the token buckets for the quotas that apply to the regions it hosts, and there is no cluster-wide coordinator metering a user's aggregate rate across all servers. In practice a user's traffic is spread across the RegionServers holding the regions they touch, so a quota expressed as '500 req/s' behaves as a per-server ceiling that approximates a global limit only when load is evenly distributed. This is why region balance and row-key design interact directly with throttling: a hotspotted table whose traffic concentrates on one region — and therefore one RegionServer — can trip that server's bucket while the rest of the cluster sits idle, producing throttling exceptions even though aggregate demand is well under the nominal quota. The mental model to keep is 'a bucket per applicable quota per server,' which explains both why throttling protects each server's local resources effectively and why it is not a precise global rate limiter.
End-to-end flow
Trace a throttled workload. An operator sets a quota: user batch may perform 500 requests per second on table events. The Master writes it to hbase:quota; within a refresh interval, every RegionServer hosting events regions loads the definition and instantiates a 500 req/s token bucket for that user-table pair.
The batch job starts issuing puts. Each put arrives at a RegionServer's RPC handler; before execution, the throttler looks up the applicable buckets — the user-table request-count bucket, plus any user-level and table-level quotas — and estimates the cost (one request, a known byte size). If the buckets have tokens, it reserves them and the put proceeds; after the put completes, actual bytes are reconciled against the reservation. As long as the job stays under 500 req/s, tokens refill fast enough that it never blocks, and its burst allowance absorbs brief spikes.
Now the job accelerates to 2000 req/s. The bucket drains its burst capacity in a fraction of a second, then refills at only 500 tokens/s. Requests arriving faster than that cannot acquire tokens, so the RegionServer rejects them with RpcThrottlingException(waitInterval=...). The HBase client catches this, honors the wait interval, and retries after backing off — so the job self-throttles to its 500 req/s contract without the operator killing it, and every other tenant on those RegionServers keeps its handlers and cache share. The greedy workload slows down; the neighbors do not.
Consider a scan instead of a put. A scan of unknown size arrives; the throttler estimates its byte cost, checks the read-size bucket, and reserves the estimate. The scan runs and returns 8MB when the estimate assumed 4MB — so on completion the throttler charges the extra 4MB against the bucket, meaning the next request feels the true cost. Over many operations this estimate-then-reconcile loop keeps the byte-rate honest even though individual costs are unknown in advance. Meanwhile a separate table has grown past its 10TB space quota with a NO_WRITES policy; its RegionServers reject new writes to that table until compactions and TTL expiry bring it back under the limit, while reads and the rest of the cluster continue unaffected.
Observe how the two quota families compose during an incident. A batch tenant is both rate-limited (500 req/s on its table) and writing into a namespace approaching its space quota. As the tenant accelerates, throttle quotas cap its request rate first, smoothing the load the RegionServers absorb; independently, as its cumulative data crosses the namespace space limit, the space-quota violation policy kicks in and begins rejecting new inserts regardless of rate. The two mechanisms address different axes — instantaneous pressure versus total footprint — and an operator debugging 'why are writes failing?' must distinguish an RpcThrottlingException (slow down and retry; capacity exists) from a space-quota rejection (you are out of your allotted size; delete data or raise the quota). Surfacing both as distinct, monitored signals is what keeps a multi-tenant cluster diagnosable: throttling tells you a tenant is going too fast, and the space policy tells you a tenant has grown too big, and the remedies for the two are entirely different.