Keeping the Prompt Cache Warm in Agent Loops: Prefix Breakers, Breakpoints, TTL Gaps and Reading the Usage Fields

Why agent loops are the best and worst case for caching

An agent loop sends the same large prefix again and again: tool definitions, a long system prompt, and a conversation that grows by a tool call and a tool result each turn. Prompt caching lets a provider reuse the processed prefix instead of reading it again, and charges much less for the reused part; LLM prompt caching explains the general mechanism, and this guide is about what agent loops do to it. On the Claude API a cache read costs 0.1 times the base input price, while writing a five-minute cache entry costs 1.25 times and a one-hour entry two times. OpenAI documents the same structure for GPT-5.6 and later, with a 30-minute default lifetime at 1.25 times and reads at 0.1 times, and gives the arithmetic: one write and nine full reads cost 2.15 times the input of one request, against ten times without caching.

The same loop is also easy to break. Caches match an exact prefix. One changed token near the start of the request makes everything after it new, and an agent framework adds plenty of things that change: timestamps, tool lists assembled at run time, compaction, and long pauses for human approval.

How the cache matches, in one paragraph

The Claude API builds the prefix in a fixed order, tools, then system, then messages, and writes a cache entry only at a block marked as a breakpoint (up to four, or one placed automatically on the last block). On the next request it checks for an entry at each breakpoint and walks back at most 20 blocks looking for one an earlier request wrote. It does not discover stable content behind a breakpoint that changed. Entries live for five minutes by default and every read refreshes them. Prefixes below a model-specific minimum length are not cached at all, silently. OpenAI’s recent models also match the longest cached prefix at breakpoints, report reads as cached_tokens, and round that count down to a multiple of 128. Gemini enables implicit caching by default on its recent models. The details differ; the rule that one early change invalidates everything after it does not.

The usual prefix breakers

Worked example

The runnable audit replays a ten-request refunds-agent loop against a simplified model of the documented behaviour: four tool definitions of 1,400 tokens, an 1,800-token system prompt, 550 tokens of history per turn, a breakpoint after the system prompt and one on the last block, five-minute entries, 1.25 for writes and 0.1 for reads. It is a teaching model built from the documentation, not a provider, and its token counts are invented.

The timestamp row is the expensive mistake: every request writes a new entry and never reads one, so caching costs a quarter more than not caching. The tool-order row costs one full rewrite: request 5 diverges at block 0 and reads nothing, and the loop recovers from request 6. Compaction keeps the static prefix because a breakpoint sits on it; request 6 reads 3,200 tokens of tools and system prompt and writes only the summary and what follows. Without that second breakpoint it would read nothing.

Finding the breaker

Log the blocks of consecutive requests, or at least a hash per block, and compare them. The first differing block is where the cache stops helping.

Illustrative python, checked for syntax; adapt names and storage to your system.

def first_divergence(prev: list, cur: list):
    """Index of the first differing block, or None if cur only extends prev."""
    for i, (a, b) in enumerate(zip(prev, cur)):
        if a != b:
            return i
    return None if len(cur) >= len(prev) else len(cur)

Run it on a recorded loop in CI. A test that fails when request 2 diverges before the end of request 1 catches a timestamp or an unsorted tool list before it reaches production.

Reading the usage fields

Measure with the provider’s own counters rather than estimates. On the Claude API, cache_creation_input_tokens and cache_read_input_tokens report writes and reads; if both are zero, the request was not cached, often because the prefix was under the model’s minimum. On the OpenAI Responses API, usage.input_tokens_details.cached_tokens reports reads and cache_write_tokens reports writes. Compute the hit rate as cached tokens over input tokens per loop, per tenant and per day, and alert when it falls; a deploy that adds a timestamp shows up as a step change the same hour.

Failure modes

• Assuming caching is on because a marker was sent, while the prefix is below the minimum and nothing is cached.
• Breakpoint placed on a block that changes every request, which pays for a write each time.
• A conversation that grows more than 20 blocks between writes, so the lookback no longer finds the previous entry.
• Choosing longer lifetimes everywhere; one-hour entries cost more to write and only pay off when pauses are common.
• Mixing tenants into one prefix to raise the hit rate, which shares cached context across customers you meant to separate.

Verification

Record a real loop in staging, replay it twice, and check that from the second request onward the read counter covers at least the tools and system prompt. Repeat after every change to the agent framework, the tool registry or the prompt template, and keep the hit rate on the cost dashboard next to the token count.

Practice and related reading

Further reading:

• LLM prompt caching
• Caching in ADK
• LLM pricing models: cache writes and reads

Sources and scope

Checked against these sources on 22 October 2026. Examples and figures are illustrative, not measurements.

• Claude API documentation: Prompt caching (breakpoints, 20-block lookback, TTL, invalidation table, usage fields)
• OpenAI API documentation: Prompt caching (minimum length, lifetimes, cached_tokens)
• Gemini API documentation: Context caching (implicit caching)