Impala + HBase Integration, in depth: table mapping, row-key pushdown, reading EXPLAIN for SCAN HBASE, joins with Parquet, writes and when to choose Kudu instead

How the pieces fit

Impala does not have its own copy of HBase data. An Impala table over HBase is an external table in the Hive Metastore whose definition says: these SQL columns come from this HBase table, mapped to these column families and qualifiers. When a query arrives, the coordinator reads that mapping through the catalog service, asks HBase for the table's region boundaries, and turns the scan into one scan range per region. Executor daemons then act as ordinary HBase clients, opening a Scan against each RegionServer with whatever start key, stop key and filters the planner could derive.

Rows come back as HBase cells. Impala decodes each mapped cell into a typed column value, then does everything else, including non-pushable predicates, joins, aggregation and sorting, in its own engine. Unlike HDFS scans, the data does not live on Impala's local disks, so every byte crosses the network from a RegionServer, and every scan competes with the operational traffic that HBase exists to serve.

Mapping a table

Impala's CREATE TABLE cannot attach a storage handler, so HBase-backed tables are created in Hive and then made visible to Impala with INVALIDATE METADATA. The mapping string lists, in column order, the source of each SQL column: :key for the row key, family:qualifier for everything else.

-- In Hive (beeline), not Impala: Impala's CREATE TABLE cannot attach the HBase storage handler.
CREATE EXTERNAL TABLE customers_hb (
    rowkey        STRING,      -- must be STRING for key pushdown
    email         STRING,
    country       STRING,
    plan          STRING,
    signup_day    STRING,      -- 'YYYY-MM-DD' kept as STRING so filters can push down
    lifetime_usd  BIGINT,
    logins        BIGINT
)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  "hbase.columns.mapping" =
  ":key,p:email,p:country,p:plan,p:signup_day,m:lifetime_usd#b,m:logins#b"
)
TBLPROPERTIES ("hbase.table.name" = "customers");

-- Then in impala-shell:
INVALIDATE METADATA customers_hb;

Three choices in that DDL are deliberate. The row key is STRING, because Impala can only turn key predicates into key ranges when the key column is STRING. Columns you will filter on are STRING, because only STRING non-key columns can become HBase-side filters. The numeric counters use the #b suffix, which maps them as binary-encoded values: smaller in storage and suitable when another application writes them with HBase's Bytes encoding, but any predicate on them is evaluated in Impala after the row arrives. Make sure the mapping matches how the writer actually encodes each cell; a mismatch produces NULLs or garbage, not errors.

The table is external. Dropping it in Impala removes only the metadata; the HBase table and its data remain. The impala service user needs HBase permissions on the table, granted in the HBase shell.

Which predicates reach HBase

This is the part that decides whether a query takes milliseconds or hours. The planner has three outcomes for a predicate on an HBase table.

Server-side filters are cheaper than shipping rows, but they still read every row in the scanned range on the RegionServer. A filter without a key range is a full scan that happens to return less data. The only way to make a query cheap is a narrow key range. For background on HBase filter semantics, see HBase filters.

Reading EXPLAIN for SCAN HBASE

Always check the plan. The SCAN HBASE node shows exactly what was pushed down.

-- Point lookup: becomes a start/stop key, touches one region.
EXPLAIN SELECT * FROM customers_hb WHERE rowkey = 'EU#000042';
|  00:SCAN HBASE [default.customers_hb]
|     start key: EU#000042
|     stop key: EU#000042\0

-- Prefix range plus a string filter: a bounded scan with a server-side filter.
EXPLAIN SELECT rowkey, email FROM customers_hb
WHERE rowkey >= 'EU#' AND rowkey < 'EU$' AND plan = 'pro';
|  00:SCAN HBASE [default.customers_hb]
|     start key: EU#
|     stop key: EU$
|     hbase filters: p:plan EQUAL 'pro'

-- Non-string predicate: no pushdown, every row comes back to Impala first.
EXPLAIN SELECT count(*) FROM customers_hb WHERE logins > 100;
|  00:SCAN HBASE [default.customers_hb]
|     predicates: logins > 100

Read it as a checklist. Start key and stop key present means the scan is bounded. An hbase filters line means work moved into the RegionServer. A predicates line means Impala does the work after fetching every row in range. If you expected a bound and see only predicates, the key is probably not STRING, the predicate uses OR or IN, or the comparison is against a non-constant expression. For an IN list of keys, rewriting as a UNION ALL of equality lookups often turns one full scan into several point gets. More on plan reading in Impala query plans.

Row-key design for SQL access

HBase row keys are designed for the access pattern, and SQL is just another access pattern. Because Impala can only exploit the key as a sorted string range, put the dimension you filter on most at the front, in a sortable string form. The example uses a region prefix plus a zero-padded customer number, EU#000042, so all European customers form one contiguous range and point lookups stay exact. Zero-padding matters: as strings, 42 sorts after 100.

Salting, the usual cure for write hotspots, works against SQL. A key prefixed by a hash bucket spreads writes but splits every logical range across all buckets, and since Impala cannot turn OR into ranges, a salted range query becomes a full scan unless you write one range per bucket and union them. If heavy SQL ranges matter, consider a secondary index table whose key is the SQL filter dimension, maintained by the writing application.

Joins with Parquet and writes

The most valuable use of the integration is joining a large columnar fact table with current state that only HBase holds. Keep the HBase side small and bounded, and make it the side Impala builds a hash table from, so the big table streams past it.

-- Fact data in Parquet, current customer state in HBase.
SELECT STRAIGHT_JOIN e.country, c.plan, count(*) AS events
FROM events_parquet e
JOIN /* +broadcast */ (
    SELECT rowkey, plan
    FROM customers_hb
    WHERE rowkey >= 'EU#' AND rowkey < 'EU$'      -- bound the HBase side
) c ON e.customer_key = c.rowkey
WHERE e.event_day = '2026-10-01'
GROUP BY e.country, c.plan;

-- Row-at-a-time writes go through the HBase client as Puts.
INSERT INTO customers_hb (rowkey, email, country, plan, signup_day, lifetime_usd, logins)
VALUES ('EU#000043', 'b@example.com', 'DE', 'free', '2026-10-02', 0, 1);

Cardinality estimates for HBase tables are rough, because there are no Parquet-style column statistics to read, so check the join order in EXPLAIN and use hints when the planner chooses badly. Never let the large Parquet table become the build side against an unbounded HBase probe.

Writes are supported with INSERT ... VALUES and INSERT ... SELECT, which become HBase Puts. Several statements are not: INSERT OVERWRITE, LOAD DATA, UPDATE and CREATE TABLE LIKE are unavailable, and complex types and TABLESAMPLE are unsupported. HBase semantics leak through: an INSERT with an existing key overwrites the cells you supply, which is effectively an upsert, and duplicate keys within one INSERT ... SELECT leave only one version. For bulk loads, HBase's own bulk-load path is far faster than SQL inserts.

Worked example: a support lookup that stays fast

A support tool shows an agent the customer's current plan from HBase next to their last thirty days of events from Parquet. The first version looked customers up with WHERE rowkey IN (...), and EXPLAIN showed only a predicates line: because IN is not converted to key ranges, every region was scanned on each page load, taking seconds under load and adding latency to the online API. The fix was to issue a single equality on rowkey per customer and bound the HBase side to that one key before joining to Parquet. A separate change, rebuilding keys as region prefix plus zero-padded number, made the regional reports range-scannable too. EXPLAIN then showed a start and stop key on one row, and the page dropped to the cost of a point get plus a partitioned Parquet scan.

Tuning and protecting the cluster

Two query options map to HBase Scan settings. HBASE_CACHING sets how many rows a scanner fetches per round trip: larger values mean fewer RPCs and more RegionServer memory per scanner. HBASE_CACHE_BLOCKS controls whether blocks read by the scan are added to the RegionServer block cache; turn it off for large analytic scans so a one-off query does not evict the hot working set that latency-sensitive applications depend on.

Protect the operational workload explicitly. Use HBase quotas or a separate RegionServer group for the analytic user if the table also serves online traffic. Use Impala admission control to cap concurrent queries on HBase tables. Prefer snapshots exported to Parquet for heavy recurring analytics: run SQL against the copy, and keep live HBase queries for fresh lookups.

Failure modes

• Accidental full scans. An integer key, an IN list or an OR on the key quietly scans every region. Always EXPLAIN before scheduling a query.
• Encoding mismatch. A column mapped as string but written as binary, or the reverse, decodes to NULL or nonsense without an error.
• Stale metadata. Changing the Hive definition without INVALIDATE METADATA leaves Impala on the old mapping.
• Block-cache pollution. A large scan with cache blocks on evicts hot data and raises latency for online clients.
• Scanner timeouts. Slow downstream operators let HBase scanner leases expire mid-query; lower HBASE_CACHING or reduce the work per scanned row.
• Version surprises. Impala reads the latest cell version; history kept in older versions is invisible to SQL.

When to use something else

Impala over HBase is right for joining a few current rows or a bounded key range with columnar history. It is wrong for large aggregations over the whole HBase table, which should run against a Parquet or Iceberg copy. If you need SQL with updates and fast analytic scans on the same mutable data, Kudu was built for that combination, as described in Impala and Kudu. If you need SQL with secondary indexes on HBase itself for transactional access, Apache Phoenix fits better.

What to do next

1. List the SQL queries you will run against HBase and the key range each one needs.
2. Create the table in Hive with a STRING key, STRING filter columns and correct #b mappings, then INVALIDATE METADATA in Impala.
3. Run EXPLAIN on every query and confirm a start and stop key appear.
4. Rewrite key IN lists as UNION ALL of equality lookups where the list is short.
5. Set HBASE_CACHE_BLOCKS to false and size HBASE_CACHING for analytic sessions.
6. Move recurring heavy aggregations to a Parquet or Iceberg copy, or evaluate Kudu for mutable analytic data.