Why architecture matters here

Hive fails on metastore bottlenecks, bad partitioning, missed stats, and LLAP misconfig. Architecture matters because engine + metastore + storage decide throughput.

Advertisement

The architecture: every piece explained

The top strip is the request path. Client JDBC/CLI. HS2 lifecycle. Metastore schema + partitions. Execution engine Tez / MR / Spark.

The middle row is acceleration. LLAP daemons long-lived + cached. ACID tables delta + compactor. File formats ORC / Parquet. Ranger + Kerberos security.

The lower rows are ops. Partitioning + bucketing. Statistics + CBO. Ops — metastore backup + upgrade + concurrent writers.

Hive — metastore + HS2 + Tez + LLAP + ACID tablesSQL on Hadoop, the persistent classicClient (JDBC/CLI)SQLHiveServer2 (HS2)query lifecycleMetastoreschema + partitionsExecution engineTez / MR / SparkLLAP daemonslong-lived, cachedACID tablesdelta + compactorFile formatsORC / ParquetRanger + KerberossecurityPartitioning + bucketingprune + joinStatistics + CBOcost-basedOps — metastore backup + upgrade + concurrent writersacceleratemanageformatauthorizepruneoptimizeoptimizeoperateoperate
Hive query lifecycle with metastore, HS2, Tez, LLAP.
Advertisement

End-to-end flow

End-to-end: SQL query lands on HS2. Metastore consulted for partitions. Tez plan produced with CBO using stats. LLAP daemons serve cached hot data. ORC files scanned; predicate pushdown active. Result returned. ACID transaction commits via delta files; compactor rewrites in background.