Why architecture matters here
Serverless platforms differ most in what they make impossible. Functions-as-a-service (Lambda, Cloud Functions gen1) made long-lived processes, custom runtimes, and in-process concurrency impossible; Kubernetes made nothing impossible and therefore made you operate everything. Cloud Run's architecture matters because of the specific line it draws: the container contract (listen on $PORT, be stateless, tolerate being killed) is all you owe the platform, and in exchange the platform owns capacity, placement, TLS, scaling, and rollout mechanics.
Three consequences of the design shape every production decision. First, in-instance concurrency changes the economics: an instance handling 80 concurrent requests costs the same as one handling 1, so IO-bound services (the majority — they wait on databases and APIs) are dramatically cheaper than on per-request-instance platforms, and connection pools amortize across requests instead of being re-established per invocation. Second, scale-from-zero is a latency contract you opt into: the first request after idle pays container pull (cached), sandbox start, and your app's boot time. A JVM app that takes 20 seconds to start is architecturally wrong for scale-to-zero and architecturally fine with min-instances=2. Third, CPU throttling outside requests (in the default request-based billing mode) means background threads, async queues flushed after the response, and in-process cron silently starve — work must happen during a request, or you switch to instance-based billing / always-on CPU and pay for it.
Teams that internalize these three constraints get a platform that removes an entire ops discipline; teams that fight them — sneaking in background daemons, assuming local disk persists, treating an instance as a pet — rediscover them as production incidents.
The architecture: every piece explained
Top row: the request path. Traffic enters at the Google Front End / global load balancer — anycast IPs, TLS termination at the edge, DDoS absorption — optionally fronted by your own external HTTPS LB for custom routing, Cloud Armor policies, and CDN. Run ingress maps host and path to a service and enforces ingress settings (all / internal-only / internal-plus-LB) and IAM: with --no-allow-unauthenticated, the platform validates the caller's identity token before your container ever sees the request — authentication as infrastructure, not middleware.
The activator path exists for the scale-from-zero moment: when a service has no warm instances, requests are buffered by the activator while the autoscaler starts instances; once capacity exists, the activator steps out of the hot path. The autoscaler's control loop is concurrency-first: target instances ≈ in-flight requests ÷ per-instance concurrency, smoothed over a short window, bounded by min and max instances, with CPU utilization as a secondary signal for CPU-bound concurrency-1 services.
Middle row: the compute plane. The instance pool holds your container replicas, each wrapped in a gVisor sandbox (first generation) or a microVM (second generation execution environment — full Linux syscall compatibility, faster file IO, required for GPUs and some kernel features), scheduled onto Borg alongside the rest of Google. Each deploy creates an immutable revision — image digest plus configuration (env vars, memory/CPU, concurrency, connections) — and traffic splitting routes percentages across revisions or pins tag-based preview URLs, which is the entire canary/rollback story: shift 5%, watch, shift 100%, or snap back instantly.
Bottom rows: networking and integrations. Direct VPC egress (or the older serverless VPC connector) gives instances private-IP paths to Cloud SQL, Memorystore, and internal services. Inbound events arrive as HTTP: Pub/Sub push subscriptions with OIDC-authenticated delivery, Eventarc routing audit-log and custom events, Cloud Tasks for rate-controlled queues, and Cloud Scheduler for cron — Cloud Run services and jobs (the batch variant) are the universal HTTP target. The ops strip carries the knobs that decide production behavior: min instances against cold starts, startup and liveness probes, concurrency tuning, and max-instance caps as the cost circuit breaker.
End-to-end flow
Walk a deploy and a traffic spike end to end. You push an image to Artifact Registry and run gcloud run deploy api --image=...:v42 --concurrency=64 --min-instances=1 --max-instances=200. Cloud Run creates revision api-00042, starts one instance (min-instances), runs the startup probe against it, and only then shifts traffic. You choose --traffic=api-00041=90,api-00042=10 for a canary. The revision is immutable: the exact image digest and config are pinned, so rollback is a routing change, not a rebuild.
Now the spike: a marketing email lands and traffic jumps from 20 to 3,000 requests per second. In-flight requests per instance blow past the concurrency target of 64; the autoscaler computes it needs ~47 instances and starts them in waves. Each new instance pays the cold-start sequence — image layers (mostly cached on the node), microVM start (hundreds of milliseconds), then your app boot (the part you control: 300ms for Go, 3–15s for an unoptimized JVM). During the ramp, requests queue briefly at the ingress rather than overwhelming existing instances beyond their declared concurrency. Within tens of seconds the pool is sized to the new load; when the spike ends, instances drain — each finishes its in-flight requests, receives SIGTERM, and gets up to ten seconds (configurable) to close connections and flush.
Meanwhile the event side keeps flowing: an order-processing Pub/Sub subscription pushes messages as authenticated HTTP POSTs to the same service. Push backoff is automatic — if the service returns errors or slows down, Pub/Sub retries with exponential backoff and eventually parks messages in the dead-letter topic after the configured attempts. A Cloud SQL query in the handler travels over direct VPC egress on private IP; the connection pool (sized to instances × pool-per-instance, watched against the database's max_connections) persists across requests because the instance persists.
The failure case is equally scripted: revision v42 has a memory leak; instances hit their 512MB limit and are killed and replaced while error rates on the 10% canary slice climb. The alert fires; --traffic=api-00041=100 restores the old revision in seconds. No cluster, no drain cordons, no pod evictions — routing did all of it.