Why it matters

Interactive apps require streaming. Users won't wait 30 seconds for a full response when they can see incremental output starting in a second. This is table stakes for chat UX.

Advertisement

The architecture

Model streaming: LLM APIs support streaming (chunked completion). ADK Java uses async streaming interfaces to receive tokens as they arrive.

Runner streaming: the runner emits events as they happen — tool calls, tool results, generation tokens, final response.

Streaming pipelineModel streamtokens flow inRunner eventsaggregate + emitClient streamSSE/WS/gRPCBackpressure required: if client slow, model output must throttle or buffer
Streaming from model to client.
Advertisement

How it works end to end

Server-Sent Events (SSE): simple, HTTP-based. Fits most web UIs. One-way from server to client. Good default.

WebSocket: bidirectional. Needed if client sends more messages mid-generation (e.g., cancel).

gRPC streaming: efficient for server-to-server. Good for backend microservices.

Backpressure: if client can't keep up with tokens, slow production. Otherwise memory buildup.