All 40 articles, sorted alphabetically
Hive ACID Transactions
How Hive supports ACID transactions with delta files and lock manager, enabling INSERT, UPDATE, DELETE, and MERGE.
Read article →Hive Bucketing
How Hive bucketing hashes rows into a fixed number of files, and when it enables sort-merge bucket joins for big-to-big joins.
Read article →Hive Cost-Based Optimizer (Calcite)
Choose plan based on stats.
Read article →Hive ACID compaction architecture
Deep-dive on Hive ACID compaction: because updates and deletes are written as delta directories on immutable storage, reads amplify and space leaks un…
Read article →Hive Deprecated Features
Indexes, MR engine, some SerDes.
Read article →Hive dynamic partitioning
Deep-dive on Hive dynamic partitioning: static vs dynamic keys and trailing-column binding, DISTRIBUTE BY for file-count control, max-partition guardr…
Read article →HiveServer2 architecture
Deep-dive on HiveServer2: Thrift transports and session/operation managers, async execution over Tez session pools and LLAP, result fetch and spooling…
Read article →Apache Iceberg -- the open table format for the lakehouse
Deep-dive on Apache Iceberg: layered metadata (catalog/manifest/data files), immutable snapshots, ACID via atomic pointer swaps, time travel, hidden p…
Read article →Hive LLAP
How LLAP (Live Long And Process) makes Hive interactive by running long-lived daemons that cache data and skip container startup entirely.
Read article →Hive Materialized Views
Precomputed aggregates + auto-rewrite.
Read article →Hive Metastore
How the Hive Metastore stores tables, partitions, and statistics, and why it has become the de facto schema registry for Spark, Presto, Impala, and Tr…
Read article →Hive Monitoring
HS2 metrics + query logs.
Read article →Hive Cost-Based Optimizer
How Hive's cost-based optimizer chooses join orders, join strategies, and plan alternatives based on table and column statistics.
Read article →ORC format architecture
Deep-dive on ORC internals: file/stripe/stream layout, RLEv2 and dictionary encodings, row-group indexes with seek positions, min/max stats and bloom …
Read article →Apache Hive Overview
What Hive is, why SQL over HDFS mattered, and where Hive fits in modern data platforms alongside Spark, Presto, and Impala.
Read article →Parquet Format
The internal structure of Parquet: row groups, column chunks, page indexes, dictionary encoding, and cross-engine compatibility.
Read article →Hive Predicate Pushdown
Filter at storage layer. ORC/Parquet stats.
Read article →Hive replication architecture
Deep-dive on Hive replication: event-driven incremental replication over the notification log, bootstrap and checkpointed cycles, DistCp data movement…
Read article →Hive skew join optimization architecture
Deep-dive on Hive skew joins: why data skew defeats parallelism, runtime vs compile-time skew join, the hive.skewjoin.key threshold, diverting hot key…
Read article →Hive small-file problem
Deep-dive on the small-file problem: sources (streaming, dynamic partitioning, appends), the three-layer cost (metadata, task overhead, read amplifica…
Read article →Hive on Tez architecture
Deep-dive on Tez execution for Hive: vertices and typed edges vs MapReduce chains, the per-session application master, YARN container reuse, runtime a…
Read article →Hive UDFs and UDAFs
Deep-dive on Hive user-defined functions: UDF/GenericUDF/UDAF/UDTF types, ObjectInspectors, vectorized implementations vs row-mode fallback, UDAF part…
Read article →Hive UDFs
How to write Hive UDFs, UDAFs, and UDTFs, when to use them, and the performance and safety implications.
Read article →Hive Vectorization
Process batches of rows. Massive perf.
Read article →Hive View Types
The main Hive view types: virtual (logical), materialized (physical), and their trade-offs.
Read article →Hive Window Functions
ROW_NUMBER, RANK, LAG, LEAD.
Read article →Impala Admission Control
How Impala admission control queues queries when resources are constrained, and how to configure per-pool limits.
Read article →Impala Architecture
The three daemon types in Impala, how they cooperate, and what each is responsible for.
Read article →Impala catalog and statestore architecture
Deep-dive on Impala metadata: catalogd as single writer over the Hive Metastore, statestored pub/sub fan-out, coordinator caches and versions, REFRESH…
Read article →Impala code generation architecture
Deep-dive on Impala runtime code generation: why interpretation is slow at scale, cross-compiling the C++ runtime to LLVM bitcode, building per-query …
Read article →Impala data cache architecture
Deep-dive on the Impala data cache: per-daemon local NVMe caching of remote byte ranges keyed by file and offset, the hit/miss/insert read-through pat…
Read article →Impala + Kudu
How Impala works with Apache Kudu to give both fast scans and low-latency updates, and when to choose Kudu over HDFS.
Read article →Impala Memory Limits
Per-query + per-node caps.
Read article →Impala query execution architecture
Deep-dive on Impala's MPP execution: coordinator planning, statestore and catalog daemons, admission control pools, pipelined…
Read article →Impala result spooling architecture
Deep-dive on Impala result spooling: why coupling execution to client fetch speed pins cluster resources, how the coordinator buffers results in memor…
Read article →Impala Runtime Filters
Dynamic filter propagation.
Read article →Impala spill-to-disk architecture
Deep-dive on Impala spill-to-disk: how partitioned hash joins, aggregations, and sorts spill a victim partition to scratch disk when their memory rese…
Read article →Impala statestore architecture
Deep-dive on the Impala statestore: a lightweight, soft-state publish/subscribe broker that disseminates cluster membership and catalog metadata acros…
Read article →Impala Table Statistics
How Impala uses table and column statistics for the cost-based optimizer, and how to keep stats fresh.
Read article →Impala vs Hive
The trade-offs between Impala and Hive: latency, concurrency, workload compatibility, ecosystem fit.
Read article →