Why architecture matters here

Impala fails on stale statistics (bad plans), concurrency (query storm), and metadata refresh (schema changes not seen). Architecture matters because statestore + catalogd + admission control decide health.

Advertisement

The architecture: every piece explained

The top strip is the request path. Client (SQL) connects to any coordinator. Frontend (Java) parses and plans distributed fragments. Backend (C++) executes on all nodes. LLVM codegen specializes hot loops for the actual data types.

The middle row is coordination. Statestore broadcasts membership + metadata. Catalogd pulls from Hive metastore + caches. Admission control gates concurrent queries via pools. Result cache serves repeat queries.

The lower rows are practice. Column formats — Parquet, Kudu, Iceberg. Observability — query profiles + history. Ops — cluster tuning + concurrency + statistics.

Impala — frontend + backend + statestore + catalogd + LLVM codegenMPP SQL on HDFS/Hive tablesClient (SQL)impala-shell / BIFrontend (Java)parse + planBackend (C++)execute fragmentsLLVM codegenruntime specializationStatestoremembership + metadataCatalogdHive metastore proxyAdmission controlresource poolsResult cachehot queriesColumn formatsParquet + Kudu + IcebergObservabilityprofiles + query histOps — cluster tuning + concurrency + statisticspropagatesyncgatecachereadprofileprofileoperateoperate
Impala MPP execution with statestore + catalog + codegen.
Advertisement

End-to-end flow

End-to-end: analyst runs SELECT on Parquet. Frontend plans; codegen produces JIT-specialized code. Backend executes on all impalads. Statestore keeps membership in sync. Admission control gates additional queries. Result cache hits skip execution.