Perplexity filtering
Small model scores each token's perplexity in context. Drop lowest-perplexity tokens (most predictable → least informative).
Advertisement
Ratio control
Target compression 2x/5x/10x. Coarser compression = more loss. Task-specific sweet spot.
Advertisement
What survives
Named entities, numbers, key verbs. What drops: filler words, redundant phrasing, easy-to-predict grammatical structure.