Why architecture matters here
Agents concentrate three kinds of production risk that ordinary stateless services don't, and deployment architecture is where each is either handled or hardcoded. Conversations are state with a lifespan: a session spans minutes to months, must survive replica churn and deploys, and is consulted on every turn — so where sessions live determines whether you can scale horizontally at all. The notebook default (in-process memory) is precisely the thing that cannot ship; the architecture's job is to make swapping it out a configuration change rather than a rewrite, which is what ADK's service interfaces buy. Teams that instead let session assumptions leak into agent code (module-level caches, local files) rediscover the problem as 'we can only run one replica'.
Agents act, so identity is architecture: a tool-calling agent is a principal that reads databases, issues refunds, and calls internal APIs. Production deployment means the agent runs as a dedicated service account with exactly the permissions its tools need — per agent, not one god-account for the platform — with secrets injected from a manager at the tool boundary and never present in prompts, session state, or code. The blast radius of a prompt-injected agent is precisely the IAM policy of its service account; that sentence should be written on the deployment review template.
Behavior versions like code but drifts like data: a prompt tweak, a tool description edit, or a model upgrade all change behavior without changing 'the code' in the traditional sense. Deployment architecture must therefore carry the whole behavioral bundle — agent config, prompts, tools, model pins — as one versioned artifact, gate it with eval suites in CI, and roll it out with traffic splitting and instant rollback, because the failure mode is not a stack trace but a subtle competence regression that only canary metrics catch. Add the economics (every request costs tokens; a retry loop is a budget incident) and the operational surface of an agent is meaningfully wider than its HTTP handler suggests.
The architecture: every piece explained
Top row: the hosting ladder. adk web/adk run is the dev loop: local runner, in-memory services, the event-inspector UI that makes trajectory debugging visual. The API server (adk api_server, or get_fast_api_app() embedded in your own FastAPI app) exposes the same runner over HTTP/SSE — the artifact you containerize. Cloud Run hosts that container with request-based autoscaling: you choose concurrency, min/max instances, VPC egress, and wire your own session database; it is the path when you need custom middleware, private networking, or organizational standardization on containers. Agent Engine inverts the ownership: you hand it the agent object (deployed from code via the SDK), and the managed runtime provides sessions, scaling, identity integration, and a query API; less to operate, less to customize — the right default for teams whose differentiation is the agent, not the platform.
Middle row: the services that swap. The session service ladder: in-memory (dev) → database-backed (self-hosted Postgres/Firestore for Cloud Run) → managed Vertex sessions (Agent Engine); the agent's contract with it is the same event-append/state-fold API throughout. The artifact service: local temp files → GCS buckets with lifecycle policies. The memory service: absent → vector-backed memory bank for cross-session recall. Because callbacks, tools, and agent logic touch only the interfaces, the eval suite that passed against dev services is testing the same behavior that ships. Identity and secrets: one service account per agent deployment scoped to its tools' needs; tool credentials from Secret Manager at call time; user-level authorization carried in session state and enforced in before_agent/before_tool callbacks — the platform authenticates the caller, the callbacks authorize the action.
Bottom rows: scale and sight. Scaling is stateless-replica shaped: any instance serves any session because state lives in the services; the knobs that matter are concurrency (agent turns are IO-bound on model calls — high concurrency works until CPU-bound tools say otherwise), min-instances against cold starts, and max-instances as the cost/quota circuit breaker. Observability: OpenTelemetry spans per model call and tool call (token counts as span attributes), event streams as the audit log, and eval runs wired into CI/CD so a deploy is gated by trajectory regression suites. The ops strip carries the rest: behavioral versioning with canary traffic, per-session and per-day token budgets enforced in callbacks, model quota capacity planning, and runbooks for the agent-specific incidents (routing regression, tool outage degrading answers, cost spikes).
End-to-end flow
Walk one team's path to production. Week one: the support agent works in adk web; the engineer inspects trajectories in the event viewer, builds an eval set of 25 recorded cases. Week two: containerize — get_fast_api_app() with the session service pointed at Cloud SQL Postgres, artifacts at a GCS bucket; CI builds the image, runs the eval suite against the containerized agent with stubbed tools, pushes to Artifact Registry. Deploy to Cloud Run staging: 2 min instances, concurrency 40, service account support-agent@… granted exactly: Cloud SQL client, the order-service invoker role, Secret Manager access to two tool credentials, Vertex AI user for model calls. A smoke eval runs against staging with live tools nightly.
Production rollout is a traffic-split ritual. Release v14 (new refund instruction + one new tool) ships as a new Cloud Run revision at 10% traffic. Dashboards watch the agent-specific canary signals side by side: p95 turn latency, tool-error rate, escalation rate, tokens per resolved conversation, and the routing-accuracy sample. Two hours clean → 100%. The next Tuesday, the model provider announces a version bump; the team pins models explicitly, so nothing moves until they run the full eval suite against the new version in staging, catch a regression in the escalation suite, patch the instruction, and migrate deliberately — the eval-gated pipeline turning a surprise into a scheduled task.
Traffic grows 10× after a product launch. Because sessions are in Postgres and replicas are stateless, Cloud Run scales to 60 instances without anyone touching the agent; the things that actually need attention are the ones the architecture surfaced in advance: the Cloud SQL connection pool (bounded per instance, pooler in front), model quota (provisioned throughput purchased for the launch), and the token budget alert that fires when a prompt-injection probe sends one session into a tool-retry loop — killed by the per-session budget callback at ₹40 instead of ₹40,000. The postmortem's happiest line: the incident was a dashboard event, not an outage, because the budget was architecture, not a hope.