Why it matters

Deployment choice affects cost, ops burden, and scale. Getting it right for the workload is critical.

Advertisement

The architecture

Cloud Run: build container, deploy. Auto-scales from zero. Pay per request. Great for spiky workloads with cold-start tolerance.

GKE: full Kubernetes. Predictable cost for baseline load. Better for high-QPS or long-running sessions.

Deployment optionsCloud Runserverless, zero-to-NGKEmanaged K8sVertex AI Agent Enginemanaged agent runtimeVertex AI handles session state and scaling; Cloud Run cheapest for bursty; GKE for stable load
Three deployment targets.
Advertisement

How it works end to end

Vertex AI Agent Engine: Google's managed agent runtime. Handles session persistence, scaling, monitoring, integrates with other Vertex AI services. Trade-off: vendor lock-in.

State considerations: Cloud Run instances can be recycled; need persistent SessionService. GKE instances more stable but same principle.

Model API auth: use ADC (Application Default Credentials) for Google models. Rotate secrets for other providers.