Why architecture matters here

Context packing fails on budget overflow, wrong salience, and missing summarization. Architecture matters because each layer decides what the model actually sees.

Advertisement

The architecture: every piece explained

The top strip is inputs. Available context. Salience scoring. Summarization. Retrieval augment.

The middle row is control. Working memory. Priority ordering. Token budget. Evictions.

The lower rows are ops. Evaluation. Metrics. Ops — governance + PII + audit.

Context packing — window management + salience + summarization + working memoryfit useful information into limited contextAvailable contextsystem + history + docsSalience scoringrecent + task-relevantSummarizationcompact old turnsRetrieval augmentjust-in-time factsWorking memoryreserved regionPriority orderingsystem > pinned > relevant > recentToken budgetcapEvictionsleast salient firstEvaluationtask success + coherenceMetricscontext usage + latencyOps — governance + PII + auditreserveordercapevicttestmeasuremeasureoperateoperate
Context packing pipeline: salience + summarization + working memory.
Advertisement

End-to-end flow

End-to-end: user turn 25 in a long chat. Old turns summarized. Salience scoring picks 3 relevant recent turns + summary of older. Retrieval brings in relevant docs. Working memory holds current facts. Budget 8k. Priority ordering ensures system + pinned always in.