Why it matters

Summarization shapes long conversations. Understanding shapes design.

Advertisement

The architecture

Threshold: N turns or M tokens.

LLM summarizes oldest chunk.

Replace with summary.

Summarization flowTriggerN turns / M tokensSummarize chunkLLM summaryReplace historywith summaryTrade information vs context length; iterate summaries for very long chats
Memory summarization.
Advertisement

How it works end to end

Summarize per-N-turns.

Cascade: summary of summaries for very long.

Retain recent verbatim.