Why architecture matters here
Compaction strategy choice matters because it's one of the highest-impact per-table tuning decisions in Cassandra -- the difference between a workload performing well and badly often comes down to matching the compaction strategy to the workload. Cassandra's LSM structure (accumulating SSTables) requires compaction (merging them), and the strategy determines how -- occupying very different points in the read/write/space tradeoff. A well-matched strategy (STCS for write-heavy, LCS for read-heavy, TWCS for time-series) makes the workload perform well (the right amplification tradeoff for the workload); a mismatched strategy (e.g., STCS for read-heavy -- high read amplification; STCS for time-series TTL -- tombstone accumulation) makes it perform badly (the wrong tradeoff -- slow reads, or tombstone problems). And because it's a per-table choice, each table can be tuned for its workload. So understanding the strategies (their tradeoffs, their fit) and choosing correctly is one of the most impactful Cassandra tuning decisions, and getting it wrong is a common, significant performance problem. For operating Cassandra, understanding compaction strategies is essential to good performance.
The read/write/space tradeoff is the analytical framework, and each strategy is a different point in it. The fundamental LSM tradeoff (read vs write vs space amplification) is navigated differently by each strategy. STCS optimizes write amplification (merging similar-sized SSTables -- each piece of data rewritten relatively few times -- low write amplification) at the cost of read amplification (a row's data can be in many size-tiers -- a read checking many SSTables -- high read amplification) and space (large SSTables of partially-obsolete data linger until they have similar-sized peers to compact with -- space amplification). So STCS is write-optimized (low write amplification -- good for write-heavy) but read- and space-costly. LCS optimizes read amplification (organizing SSTables into levels of non-overlapping SSTables -- so a row is in at most one SSTable per level, and there are few levels -- a read checking few SSTables -- low read amplification) and space (the leveled structure reclaiming obsolete data efficiently -- low space amplification) at the cost of write amplification (maintaining the leveled, non-overlapping structure requires more rewriting -- high write amplification). So LCS is read-optimized (low read amplification -- good for read-heavy) but write-costly. TWCS optimizes for time-series with TTL (grouping SSTables into time windows and compacting within a window -- so each window's data, sharing a time range, expires together and the whole window's SSTable can be dropped when expired -- efficient TTL expiry) -- avoiding the tombstone accumulation that STCS/LCS hit with time-series TTL. So each strategy is a different point in the read/write/space tradeoff (STCS write-optimized, LCS read-optimized, TWCS time-series-optimized), and understanding this framework (each strategy's tradeoff) is understanding how to choose.
And the tombstone-and-time-series reality is a crucial, often-painful consideration, especially motivating TWCS. In Cassandra, deletes and TTL expiry create tombstones (markers -- the data in immutable SSTables not removed in place, reclaimed by compaction). For time-series data with TTL (a very common Cassandra pattern -- e.g., metrics, events, expiring after a retention period), STCS and LCS have a serious problem: the expiring data (and its tombstones) is spread across SSTables that also contain non-expiring data (mixed by the size-tiered or leveled compaction -- not grouped by time), so the expired data can't be efficiently dropped (it's mixed with live data in the SSTables -- the tombstones accumulating, and the expired data lingering -- causing tombstone problems: read performance degradation from scanning tombstones, and space from un-reclaimed expired data). TWCS solves this by grouping SSTables into time windows (so each SSTable contains data from one time range -- and when that time range's data all expires -- TTL -- the whole SSTable can simply be dropped -- efficient, no tombstone scanning or mixed-SSTable problem). So for time-series with TTL, TWCS is the right strategy (efficient time-based expiry -- the whole time-window SSTable dropped when expired), and using STCS/LCS instead causes tombstone problems (accumulation, read degradation, space). Understanding the tombstone/time-series issue (and TWCS as its solution) is understanding a crucial, common Cassandra consideration -- the time-series TTL workload needing TWCS.
The architecture: every piece explained
Top row: the problem and strategies. The LSM problem: writes accumulate into immutable SSTables, requiring compaction (merging them to bound read amplification and reclaim space). STCS (Size-Tiered): merges SSTables of similar size into larger ones -- write-optimized (low write amplification) but higher read amplification (data in many tiers) and space (large obsolete SSTables lingering). LCS (Leveled): organizes SSTables into levels of non-overlapping SSTables -- read-optimized (few SSTables per read) and space-efficient, but higher write amplification (maintaining the levels). TWCS (Time-Window): groups SSTables into time windows, compacting within a window -- ideal for time-series with TTL (each window's data expires together, the window's SSTable dropped efficiently).
Middle row: the amplifications and tombstones. Read amplification: the SSTables checked per read (STCS high -- data across tiers; LCS low -- few SSTables per read) -- affecting read performance. Write amplification: the rewrite cost (STCS low -- few rewrites; LCS high -- maintaining the levels) -- affecting write cost and compaction I/O. Space amplification: obsolete data lingering (STCS higher -- large obsolete SSTables; LCS lower -- efficient reclamation) -- affecting disk usage. Tombstone handling: how expiring/deleted data (tombstones) is reclaimed -- STCS/LCS mixing expiring data across SSTables (tombstone accumulation for time-series TTL), TWCS grouping by time (efficient window-based expiry) -- crucial for time-series/TTL workloads.
Bottom rows: matching and unification. Workload matching: choosing the strategy for the workload -- STCS for write-heavy (low write amplification), LCS for read-heavy (low read amplification, worth the write cost), TWCS for time-series/TTL (efficient time-based expiry) -- the per-table strategy matched to the table's workload. UCS (Unified): a newer, tunable compaction strategy (behaving like STCS or LCS via configuration -- unifying the choice into a tunable spectrum -- so the tradeoff can be tuned rather than choosing a fixed strategy) -- the modern unified approach. The ops strip: strategy choice (choosing the compaction strategy per table -- matched to the workload -- the high-impact decision), monitoring (monitoring the compaction -- SSTable counts per read (read amplification), compaction I/O (write amplification), space, tombstones -- to verify the strategy is working and catch problems), and tuning (tuning the strategy's parameters -- e.g., the size tiers for STCS, the level sizes for LCS, the time window for TWCS -- and switching strategies if the workload changes).
End-to-end flow
Trace matching strategies to workloads. A write-heavy table (high write throughput, relatively few reads -- e.g., an ingestion/logging table): STCS is chosen (write-optimized -- low write amplification, minimizing the rewrite cost for the heavy writes) -- the higher read amplification acceptable (few reads). A read-heavy table (frequent reads, needing low read latency -- e.g., a serving table): LCS is chosen (read-optimized -- low read amplification, a read touching few SSTables -- fast reads) -- the higher write amplification (maintaining the levels) accepted for the read performance. A time-series table with TTL (metrics/events expiring after a retention period -- e.g., 30-day retention): TWCS is chosen (grouping SSTables into time windows -- e.g., daily windows -- so each day's data is in its own SSTables, and when the 30-day TTL expires a day's data, the whole day's SSTable is dropped -- efficient expiry, no tombstone accumulation). Each table's strategy is matched to its workload (STCS write-heavy, LCS read-heavy, TWCS time-series TTL) -- the per-table matching optimizing each.
The mismatch and tombstone vignettes show the failure modes. A mismatch case: a read-heavy table was left on STCS (the default) -- suffering high read amplification (a read checking many SSTables across the size tiers -- slow reads). Switching it to LCS (read-optimized -- few SSTables per read) dramatically improved the read latency (the read amplification reduced) -- the strategy match fixing the read performance. A tombstone case: a time-series table with TTL was on STCS -- hitting tombstone problems (the expiring data spread across SSTables mixed with non-expiring data -- so the expired data couldn't be efficiently dropped, the tombstones accumulating -- degrading reads (scanning tombstones) and wasting space (un-reclaimed expired data)). Switching to TWCS (grouping by time window -- so each window's data expires together and the window's SSTable is dropped) solved the tombstone problem (efficient time-based expiry -- no tombstone accumulation) -- the TWCS match fixing the time-series TTL problem.
The UCS and tuning vignettes complete it. A UCS case: on newer Cassandra, the team uses UCS (the unified strategy -- tunable to behave like STCS or LCS) -- so instead of choosing a fixed strategy, they tune UCS's parameters to the desired tradeoff (more write-optimized or more read-optimized) -- the unified, tunable approach (a spectrum rather than fixed strategies). A tuning case: the team monitors the compaction (SSTable counts per read -- read amplification; compaction I/O -- write amplification; tombstones; space) -- verifying each table's strategy is working (the expected tradeoff) and catching problems (e.g., a table whose workload shifted, now mismatched -- prompting a strategy switch) -- the monitoring and tuning keeping the strategies matched. The consolidated discipline the team documents: choose the compaction strategy per table matched to the workload (STCS for write-heavy -- low write amplification; LCS for read-heavy -- low read amplification; TWCS for time-series/TTL -- efficient time-based expiry), understand the read/write/space tradeoff (each strategy a different point), use TWCS for time-series with TTL (avoiding the tombstone accumulation STCS/LCS hit), consider UCS on newer Cassandra (the tunable unified strategy), monitor the compaction (read/write/space amplification, tombstones -- verifying the strategy and catching mismatches), and tune/switch strategies as workloads change -- because the compaction strategy is a high-impact per-table decision (each strategy a different point in the read/write/space tradeoff), and matching it to the workload (STCS/LCS/TWCS for write/read/time-series) is one of the most impactful Cassandra tuning decisions.