When a streamed model turn is retried, discard everything from the abandoned attempt, hold tool calls until an attempt finishes, and key effects by meaning rather than call ids.
The reply that said everything twice
A streamed model turn is not one response. It is a sequence of events: text fragments, tool-call fragments, a finish reason. Clients render the text as it arrives and collect the tool calls as they complete. When the connection breaks halfway, a good client retries the request. The question this guide is about is what happens to everything the first attempt already delivered.
The Gemini CLI preview release v0.69.0-preview.0, published on 10 November 2026, includes pull request 30165, "handle mid-stream retries to prevent text concatenation". Its description is specific: on a mid-stream retry, partial text and tool calls from abandoned attempts were being concatenated and executed, particularly in the Agent Client Protocol mode the CLI uses to talk to editors. The fix resets the turn's accumulators on a retry event, including pending tool calls and the finish reason, and marks the next message chunk sent to the editor with a reset flag so the editor can discard its partial buffer. This is a preview release, so treat the specific behaviour as subject to change; the underlying mistake is common to any client that streams and retries, and this guide works through it with an original model rather than the CLI's code.
Replaying a broken stream
The companion uses a hand-written event list for a help-desk turn at Belgavi's Book Store. The first attempt emits a text fragment, a complete tool call to issue_refund for order ORD-7781 with id c1, and then half a sentence, "Refunding the dam", before the connection resets. The client retries. The second attempt emits a slightly different opening, a tool call with the same name and arguments but id c7, a full sentence and a finish event.
Three accumulation policies give three results:
- Naive: keep appending. The user sees "Checking order ORD-7781. Refunding the damChecking order ORD-7781. The book arrived damaged, so I have started a full refund." Two refunds were executed, because the client ran each tool call as soon as it was complete.
- Reset on retry: clear text and pending calls when the retry starts. The final text is clean. Two refunds were still executed, because the first call ran before the reset.
- Reset on retry and execute only after the attempt finishes: the text is clean and one refund runs, the one from the successful attempt.
The middle case is the one teams ship by mistake. Resetting buffers fixes what the user reads and leaves the side effect in place. The text problem is cosmetic; the tool problem is not.
Why the provider ids do not save you
It is tempting to deduplicate tool calls by their ids. That does not work across attempts. Each attempt is a separate generation, and the call identifiers are assigned per generation: c1 in the first attempt, c7 in the second, for the same intended action. Even the text differs a little, because sampling at a non-zero temperature rarely reproduces the same tokens. A retry is a new answer to the same question, not a replay of the old one.
So there are two decisions to make. Which attempt's output is the turn's output? The last complete one, always, with everything from earlier attempts discarded. And how do you stop effects from a discarded attempt from happening? You either do not run tools until the attempt has finished, or you make the tools idempotent with keys derived from the turn and the call's content rather than from provider ids.
Rule one: attempts have identity
Give every attempt a number in the client and tag every event with it. Accumulators belong to an attempt, not to the turn. When a retry begins, the previous attempt's accumulators are closed and marked abandoned. Nothing downstream reads them except for debugging.
class Turn:
def __init__(self, turn_id):
self.turn_id, self.attempt = turn_id, 0
self.text, self.calls, self.finish = [], [], None
def on_retry(self, reason):
self.attempt += 1
self.text, self.calls, self.finish = [], [], None # abandon, do not merge
self.emit_ui({"reset": True, "attempt": self.attempt, "reason": reason})
def on_text(self, fragment):
self.text.append(fragment)
self.emit_ui({"text": fragment, "attempt": self.attempt})
def on_tool_call(self, call):
self.calls.append(call) # collect, do not execute yet
def on_finish(self, reason):
self.finish = reason
return [dict(c, attempt=self.attempt) for c in self.calls] # now safe to executeThe reset event matters as much as the reset. Any consumer that renders deltas, an editor plugin, a web chat, a log tailer, has its own buffer. If it is not told to discard, it will keep the abandoned text on screen even when the client's own state is clean. Gemini CLI's fix adds exactly such a marker for the editor protocol. Your own transport needs one too: a typed event, not a convention that consumers must infer from a gap in the stream.
Rule two: execute after the attempt completes
Most agent loops already wait for the model's turn to finish before running tools, because the model may emit several calls and the loop wants all of them. Streaming tempts teams to start early, running a call the moment its arguments are complete, to save latency. That optimisation is safe only for calls that are pure reads with no side effects, or for calls whose effect is idempotent under a key the client controls.
For anything that changes state, such as a refund, an email, a case update or a file write, wait for the finish event of the attempt. If the attempt is abandoned, its calls are discarded unexecuted. The latency cost is the time between a call's completion and the end of the attempt, usually small compared with the tool's own latency.
Rule three: keys from meaning, not from transport
Even with late execution, effects can repeat: the client can crash after running a tool and before recording that it ran, and the whole turn is retried on restart. The tool needs an idempotency key, and the key must be stable across attempts. The companion derives it from the turn id, the tool name and the canonicalised arguments. For the two refund calls with ids c1 and c7, the keys are identical, so the payment service applies one refund.
import hashlib, json
def effect_key(turn_id, call):
material = json.dumps([turn_id, call["name"], call["args"]], sort_keys=True)
return hashlib.sha256(material.encode()).hexdigest()[:16]This key has a trade-off you must decide deliberately: two genuinely separate refunds of the same amount in the same turn would collide. If the domain allows that, include a step number assigned by the orchestrator, never by the model's text.
Where retries hide
Mid-stream retries are not always visible in your code. An SDK may retry a dropped stream internally; a gateway may retry on a reset; a load balancer may resend an idempotent-looking request. Read the retry behaviour of each layer between you and the model. If a layer retries a streaming request transparently and splices the second stream onto the first, you have the naive policy whether you wrote it or not. Prefer layers that surface the retry as an event, and turn transparent retries off where they cannot tell you.
Also consider what the server did with the abandoned attempt. Depending on the provider and the serving stack, generation may continue for a while after the client disconnects, and you should assume the abandoned attempt's tokens are billed unless the provider documents otherwise. A retry therefore costs at least a second generation, and a retry storm on a flaky network multiplies cost. Cap retries per turn and record each attempt's usage separately.
Testing it
The companion's tests encode the three policies on one synthetic stream and check the outcomes: the naive policy concatenates and executes twice, reset alone fixes the text but still executes twice, reset with late execution executes once and only the call from the second attempt, and the content key collapses the two provider ids into one effect. In a real client, inject failures at the transport layer, not in a mock of the model: close the socket after a tool-call event, after a text fragment that splits a word, and immediately before the finish event. For each, assert what the user sees, which tools ran and with which keys, and how many attempts were billed.
What stays deterministic
The model decides what to say and which tools to call. Everything about attempts is the client's job: numbering them, deciding that the last complete attempt wins, telling every renderer to reset, holding effects until an attempt completes and keying effects by meaning. None of that should depend on the model noticing that it is being retried; it cannot.
Run the example yourself
The three accumulation policies on one synthetic broken stream, plus the content-derived effect key. It calls no provider. Companion code on GitHub; its README lists the prerequisites, exact commands, expected output and limits.