Why architecture matters here
State design is where agent projects quietly succeed or fail, because three distinct problems masquerade as one. The record problem: what happened must be durable, ordered, and attributable — for resumption after crashes, for audit ('why did the agent say that?'), for evals (replaying real conversations), and for debugging. Event sourcing solves it: nothing mutates, every change is an event with an author, and any past moment is reconstructible. The working-set problem: what the agent needs right now — the order being discussed, the user's tier, the workflow step — must be cheap to read and write without dragging the whole history around. The state fold solves it: deltas accumulate into a current view, and instruction templating makes the view directly usable by prompts. The context problem: what the model sees each turn is a token budget, not a database — and conflating 'in the session' with 'in the prompt' is the root of both cost explosions and the mid-conversation amnesia users hate. Windowing and compaction manage the translation.
The scope system earns its place the day you have more than one user. A fact like 'prefers metric units' written to session scope dies with the conversation and gets re-learned forever; written to user: scope it follows the person; written (by bug) to app: scope it becomes everyone's preference — a one-character prefix separating personalization from a data-leak incident. Scopes are the type system of conversational state: they encode ownership and lifetime in the key itself, which is why mature teams treat the key namespace as a reviewed schema, not a scratchpad.
And the timescale split — session vs memory — is what keeps both honest. Sessions grow linearly with conversation and must stay small enough to load per turn; memory grows with the user's lifetime and must be searched, not loaded. Systems that skip the distinction either stuff everything into ever-longer sessions (cost, latency, and eventually context collapse) or build one giant vector store with no notion of a conversation (losing the coherent thread that makes multi-turn work). The architecture's claim is that turn, conversation, and lifetime are different data structures — because they are.
The architecture: every piece explained
Top row: the record and its fold. A session is identified by (app, user, session id) and holds events: user messages, agent responses, function calls/results, and state_delta actions — each immutable, ordered, authored. State is not stored as a mutable blob but computed: the fold of all deltas, materialized by the service for cheap reads. Writing happens through events — a tool sets tool_context.state['order_id'] = ..., which becomes a delta on that tool-response event — so every state change has provenance and replaying events reproduces state exactly. Scopes partition the namespace by lifetime: session-scoped working data; user: facts persisted across the user's sessions (the service handles the cross-session join); app: shared read-mostly config; temp: turn-local scratch that never persists — the right home for request-context like auth payloads.
Middle row: the services and the window. SessionService is the swap point: in-memory for dev/tests, database-backed (Postgres/Firestore) for self-hosted production, Vertex-managed for Agent Engine — same API, different durability and scale characteristics; session listing, deletion, and TTL policy live here. Context windowing is the runtime's translation of session→prompt: recent events verbatim, older history truncated or summarized, tool noise filtered — configurable, and the lever that decouples conversation length from per-turn cost. Compaction makes long sessions sustainable: periodic summarization events capture the essentials of older spans so the window can drop the originals without amnesia. MemoryService handles the lifetime axis: sessions (or their distillations) are ingested into a searchable store — keyword-based in dev, vector-backed managed memory in production — and agents query it via memory tools ('what did we decide about the Q3 budget?') rather than loading it wholesale.
Bottom rows: the ergonomics and the escape hatch. Instruction templating injects state into prompts declaratively — an instruction containing {user:tier} or {order_status} is resolved per turn from state, keeping prompts personalized without handler code. Artifacts store binary/large payloads (documents, images, reports) under names with versioning, referenced from events — the discipline that keeps a 30-page PDF out of the token stream while keeping it one tool call away. The ops strip: a state-key registry (every key: scope, writer, readers, PII class), session TTL and deletion policy wired to privacy requirements, and PII discipline — what may enter state at all, and what must stay in temp: or the secret manager.
End-to-end flow
Follow one user across the three timescales. Monday, session S1: a customer works through a billing dispute with the support agent. Turn by turn, events append; the dispute's facts accumulate in session state (dispute_id, disputed_amount, evidence_status); the resolution tool writes user:preferred_contact = 'email' when the user mentions it — a fact worth keeping beyond this thread. The conversation runs 60 turns; windowing keeps the prompt at the last ~20 events plus a compaction summary of the earlier phase ('user disputes charge 8842, invoice provided, awaiting merchant response'), so turn 60 costs the same tokens as turn 20. The full PDF invoice the user uploaded lives as an artifact; events reference it; the model sees a one-line description and can request extraction when needed.
Session S1 closes; its distillation is ingested into memory: the dispute's outcome, the user's communication preference, the merchant involved. Thursday, session S2 — a different thread, new session id, empty session state. The user opens with 'any update on that dispute?' The agent's memory tool searches ('dispute', this user), retrieves S1's distillation with the dispute id, and the agent answers with continuity the session alone could not provide — then copies dispute_id into S2's session state so the rest of this conversation reads it locally. Meanwhile {user:preferred_contact} in the instruction template means the agent offers email follow-up unprompted; the personalization came from scope design, not model magic.
The operational subplots run underneath. The temp: scope carried each request's auth context (validated user id from the API gateway) into before_agent checks without persisting it. When the user later invokes their data-deletion rights, the runbook is tractable because the architecture is: delete the user's sessions (service API), their user: keys, their memory entries, and their artifacts — four enumerable stores, one registry saying what lives where. And when an engineer investigates a mis-answered turn from S1, they replay the event log in the dev UI, watching state fold step by step until the wrong delta appears — a tool wrote evidence_status='complete' on a partial upload; the fix ships with an eval case replaying that exact session. Every capability in the story — continuity, personalization, deletion, debugging — traces to a specific layer doing its one job.