Why architecture matters here
ADK Java deployment matters because getting a Java agent to production reliably requires handling both standard JVM production concerns and agent-specific ones -- and the JVM characteristics (startup, memory) interact with agent deployment patterns. A production agent needs the standard things (packaging, config, scaling, observability, health) done well, plus agent-specific things (durable sessions for stateful conversations, concurrency for many IO-bound conversations, eval gates for behavior). And the JVM's characteristics matter: its startup time (JVMs take time to start and warm up -- JIT compilation) affects autoscaling responsiveness (new instances aren't instantly ready) and scale-to-zero (cold start latency), and its memory footprint affects density and cost. So deploying an ADK Java agent well means both production-grade deployment practices and JVM-aware handling (tuning startup and memory, choosing concurrency models suited to the workload). For teams running Java agents in production, understanding these concerns -- standard production plus agent-specific plus JVM-specific -- is what makes the difference between a reliable, efficient deployment and one with startup lag, memory bloat, or reliability gaps.
The concurrency-model choice is a key JVM-specific decision, and it's particularly relevant for the IO-bound agent workload. An agent's work is IO-bound -- each turn waits on model calls (hundreds of ms to seconds) and tool calls (databases, APIs) -- so a single agent serving many concurrent conversations must not block a thread per waiting conversation (that would exhaust threads). Java offers two solutions. Reactive (non-blocking, using reactive frameworks -- the traditional JVM answer to high-concurrency IO, with composability and backpressure but added complexity). Virtual threads (Java 21+ -- lightweight threads that don't pin OS threads, so blocking-style code -- much simpler than reactive -- scales to many concurrent conversations without the OS-thread cost). For agents (IO-bound, many concurrent conversations), virtual threads are often the better choice (the simplicity of blocking-style code with the scalability of non-blocking) -- a significant Java 21 benefit for agent workloads. Choosing the concurrency model (virtual threads for simplicity and scalability, reactive where its composability/backpressure is needed) suited to the IO-bound agent workload is a key deployment decision, and virtual threads make Java agents scale to many conversations without thread exhaustion or reactive complexity -- a JVM-specific consideration that shapes the deployment.
And the durable-session requirement is the agent-specific concern that enables the deployment's scalability and reliability. Agents are stateful (conversation state accumulates across turns -- the session), and this state must be handled correctly for production deployment. If session state lives in the instance's memory, the deployment can't scale (each instance has different state -- a conversation pinned to one instance) or survive failures (an instance crash loses the conversations' state). So production ADK Java deployment requires a durable session backend -- session state in a shared, durable store (a database) rather than instance memory -- so that any instance can serve any conversation (enabling stateless scaling -- add instances freely, any handles any conversation) and state survives instance failures (durable, recoverable). This durable-session requirement (state externalized to a durable store) is what makes the deployment scalable (stateless instances) and reliable (state survives failures) -- the agent-specific foundation for a production deployment. Understanding that durable sessions (not instance-memory state) are required for scalable, reliable agent deployment -- and using a durable session backend -- is essential to deploying ADK Java agents in production.
The architecture: every piece explained
Top row: packaging and configuration. Packaging: the deployable artifact -- a fat JAR (all dependencies bundled) or a container (the JAR in a container image) -- the standard JVM packaging. Runtime targets: where it runs -- Cloud Run (serverless containers, autoscaling), GKE (Kubernetes, more control), or VMs (traditional) -- the deployment environment. Config and secrets: externalized configuration (environment-specific settings -- model endpoints, keys -- via environment variables, config services, or Spring profiles) and secrets (credentials from a secret manager, not hardcoded) -- managed like any production service. Session backend: the durable session store (a database -- Postgres, Firestore, etc.) holding conversation state, so instances are stateless and state survives failures -- the agent-specific state foundation.
Middle row: concurrency, scaling, observability, tuning. Concurrency: virtual threads (Java 21+ -- lightweight, blocking-style, scaling IO-bound conversations without OS-thread cost) or reactive (non-blocking, composable) -- for the many-concurrent-conversation IO-bound workload. Scaling: stateless replicas (any instance serves any conversation, since state is in the durable backend) -- adding instances for capacity, autoscaling on load. Observability: Micrometer (metrics -- latency, tokens, cost, tool durations), tracing (spans per model/tool call), and structured logging -- the visibility into the agent's behavior and performance. JVM tuning: heap size (for the memory footprint), GC algorithm (for pause behavior), and startup time (JIT, class loading -- affecting autoscaling responsiveness and cold starts) -- the JVM-specific tuning affecting cost and responsiveness.
Bottom rows: lifecycle and pipeline. Health and lifecycle: readiness gates (the instance signals ready only when it can serve -- so traffic isn't routed to warming-up instances), health checks (liveness -- restart unhealthy instances), and graceful shutdown (draining in-flight requests, closing resources on shutdown -- for clean rolling deploys) -- the lifecycle handling for reliable deployment. CI/CD: the pipeline -- build (compile, package), test (unit, integration), eval gates (agent behavior tested -- trajectory evals gating the deploy), and deploy (rolling, canary) -- the automated delivery with agent behavior verification. The ops strip: startup time (the JVM startup affecting autoscaling and cold starts -- tuning it, or using techniques like AOT/native images for faster startup; min-instances to avoid cold starts for latency-sensitive agents), memory (the JVM's footprint -- heap sizing, density, cost), and cost per request (the agent's cost -- model tokens plus compute -- tracked and optimized).
End-to-end flow
Trace an ADK Java agent to production. The agent is packaged (a container image -- the fat JAR containerized), configured (externalized config -- the model endpoint and settings via environment variables, secrets from a secret manager, profiles for dev/staging/prod), and its sessions backed by a durable store (Postgres -- conversation state in the database, so instances are stateless). Deployed to Cloud Run (serverless containers, autoscaling): the agent uses virtual threads (Java 21 -- each conversation's IO-bound work on a virtual thread, so one instance serves many concurrent conversations without thread exhaustion). Under load, Cloud Run autoscales (adding instances -- stateless, so any handles any conversation, state in the durable backend). Observability flows (Micrometer metrics -- latency, tokens, cost -- to monitoring; tracing -- model and tool call spans). The deployment is scalable (stateless instances, durable sessions), observable (metrics and tracing), and handles the IO-bound concurrency (virtual threads) -- a production-grade Java agent deployment.
The startup-time and concurrency vignettes show the JVM-specific handling. A startup case: the JVM's startup time (start plus JIT warmup) means new autoscaled instances take time to be ready -- so during a sudden load spike, the new instances lag (not instantly serving). The team handles it: readiness gates (new instances signal ready only when warmed up -- so traffic isn't routed prematurely), min-instances (keeping some warm instances to absorb spikes during the startup lag), and JVM startup tuning (faster startup -- tiered compilation settings, or considering AOT/native compilation for much faster startup). The JVM startup time was managed (readiness, min-instances, tuning) so autoscaling works despite the startup lag. A concurrency case: the team chose virtual threads (over reactive) -- the agent's IO-bound conversations run on virtual threads (blocking-style code -- simple), scaling to hundreds of concurrent conversations per instance without OS-thread exhaustion (virtual threads don't pin OS threads) -- the Java 21 virtual-threads benefit for the agent workload (simplicity plus scalability), avoiding the reactive complexity.
The lifecycle and CI/CD vignettes complete it. A lifecycle case: a rolling deploy -- new instances start (readiness-gated -- serve only when ready), traffic shifts, old instances drain (graceful shutdown -- finishing in-flight conversations' turns, closing resources) -- a clean deploy with no dropped conversations (readiness plus graceful shutdown). A CI/CD case: the pipeline builds, tests (unit, integration), and runs eval gates (the agent's behavior tested -- trajectory evals verifying it takes correct actions -- gating the deploy, so a behavior regression fails the pipeline) before deploying (canary -- a small traffic percentage first, monitored, then full) -- automated delivery with agent behavior verification and safe rollout. The consolidated discipline the team documents: package as JAR/container, externalize config and secrets, use a durable session backend (stateless instances, state survives failures -- the scalability/reliability foundation), choose the concurrency model suited to the IO-bound workload (virtual threads for simplicity and scalability, reactive where needed), scale statelessly, observe with Micrometer/tracing, tune JVM startup and memory (managing the JVM's characteristics -- readiness gates and min-instances for startup lag, heap sizing for memory), handle health and lifecycle (readiness, graceful shutdown), and gate deploys on evals (agent behavior verified) -- because deploying an ADK Java agent is deploying a stateful, IO-bound JVM service, requiring production practices (packaging, config, scaling, observability), agent-specific handling (durable sessions, behavior evals), and JVM-aware tuning (startup, memory, concurrency) for a reliable, efficient production deployment.