Serverless is usually explained as 'no servers to manage', which is a billing statement dressed up as an architecture. The architectural claim is narrower and more useful: you give the platform a function and a description of what should invoke it, and the platform owns the process lifecycle -- when to start one, how many to run, when to stop them. Everything that makes serverless systems hard follows from surrendering that control. You no longer choose when a process is warm, so cold starts become a latency problem. You no longer choose how many run, so concurrency becomes the capacity unit and the blast radius. You no longer own the retry loop, so failure semantics are set by whichever event source invoked you -- and they differ, per source, in ways that decide whether a bad message is dropped, retried forever, or parked somewhere you can inspect it.
The invocation pipeline
Every serverless execution follows the same chain: an event source produces something, a trigger binds that source to a function, the platform selects or creates an execution environment, your handler runs against a deserialised event object, and a result or an error propagates back along a path that depends on how the invocation was made.
Sources fall into three families and the family determines almost everything about error handling. Synchronous sources -- an HTTP request through an API gateway, a direct invoke -- hold a connection open and expect a response; the caller sees your failure. Asynchronous sources -- an object-storage notification, an event bus, a notification topic -- hand the platform an event and return immediately; the platform owns the retry loop and the caller never learns the outcome. Poll-based sources -- queues, streams, change feeds -- are inverted: the platform runs a poller that reads batches on your behalf and invokes you with them, so batching, ordering and redelivery are properties of the source rather than of your function.
Writing a handler without knowing which family invoked it is the root cause of a large share of production serverless incidents. The same twenty lines of business logic is at-most-once behind one source and at-least-once behind another.
Cold starts — what they are and what actually removes them
An execution environment has a lifecycle: init, then some number of invokes, then shutdown. Init downloads and unpacks the deployment artifact, starts the runtime, and runs your module-level code -- imports, client construction, configuration loading. A cold start is any invocation that has to wait for init because no warm environment was available.
The cost is dominated by what you do in init, not by the platform. A small Node or Python function with lean imports starts in tens of milliseconds. A JVM function that loads a dependency-injection container and an ORM can spend seconds. The practical levers, in the order they usually pay: trim the dependency graph and the artifact size; move client construction out of the handler into module scope so it is paid once per environment rather than once per request, but keep anything slow and optional out of the critical path; prefer a lighter runtime for latency-critical paths.
Two platform features attack the remainder. Provisioned concurrency keeps a configured number of environments initialised and waiting, which converts a latency problem into a fixed bill -- you pay for the reservation whether or not it is used. Snapshot-based start takes a memory snapshot after init and restores from it, which removes most of the runtime and framework startup cost for languages where that dominates. Snapshots come with a real correctness caveat: anything captured at snapshot time is shared by every restored environment, so random seeds, connection handles and cached credentials must be re-initialised through the runtime-provided hooks or you ship duplicated entropy across thousands of executions.
Note what does not work: the 'ping it every five minutes' keep-warm trick keeps roughly one environment alive and does nothing for a burst that needs two hundred.
Concurrency is the capacity unit
In a server world you reason in requests per second against a fixed pool. In a serverless world the platform gives you one execution environment per concurrent invocation, so the number that matters is concurrency -- and by Little's law it is simply arrival rate multiplied by average duration. One hundred requests per second at 200 ms is twenty concurrent executions. The same rate at two seconds is two hundred. Reducing function duration reduces required concurrency proportionally, which is why a slow downstream dependency shows up first as a concurrency-limit alarm rather than a latency alarm.
Accounts carry a total concurrency limit -- on AWS the default is 1,000 executions per region, raisable on request -- and that pool is shared by every function in the account. This is the sharpest operational edge in serverless: an unimportant function that suddenly scales can starve a critical one, because they are drawing on the same pool. The fix is reserved concurrency, which both guarantees a floor for the function that has it and caps that function's ceiling. Setting a reservation is therefore two decisions at once, and the capping half is frequently the one you actually wanted.
Scaling is also rate-limited, not instantaneous. Platforms add environments in increments over time rather than granting a thousand at once, so a step-function traffic spike is absorbed as a burst of throttles and retries before capacity catches up. If the workload is spiky and latency-sensitive, provisioned concurrency covers the step and on-demand scaling covers the tail.
Downstream, concurrency is a weapon pointed at everything you call. Five hundred concurrent functions opening database connections will exhaust a connection pool sized for a handful of application servers. A connection proxy or a hard reserved-concurrency cap is not optional in front of a relational database.
Memory, CPU and the cost model
Function memory is a single dial that sets more than memory. CPU allocation scales with it -- on Lambda a function reaches roughly one full vCPU near 1,769 MB, and more beyond that -- and network throughput scales with it too. This produces the most counterintuitive tuning result in serverless: raising memory frequently lowers cost. Billing is memory multiplied by duration, so if doubling memory more than halves duration on a CPU-bound workload, the bill goes down and the latency improves. The only way to know is to sweep the setting against a representative payload; tools exist to automate the sweep and they pay for themselves on any function with meaningful volume.
The cost formula has two terms: a per-request charge and a compute charge in gigabyte-seconds, billed at millisecond granularity. Two consequences follow. First, idle time inside your handler is billed -- a function that sleeps waiting on a slow HTTP call is paying full freight to do nothing, which is why fan-out to a queue often beats a synchronous chain of function-to-function calls. Second, at high sustained utilisation the arithmetic turns against serverless: a container running flat out on a reserved instance is cheaper per unit of work than the same work billed per invocation. Serverless wins decisively on spiky, low-duty-cycle workloads and loses on steady high-throughput ones, and the crossover is worth computing rather than assuming.
Timeouts, retries and idempotency
Every function has a timeout ceiling -- 15 minutes on Lambda, with lower defaults elsewhere -- and hitting it is not a graceful failure: the environment is killed, so any 'clean up in a finally block' logic does not run. Set the timeout slightly above the realistic worst case, not at the maximum, because the timeout is also what bounds how long a stuck invocation consumes a concurrency slot.
Retry behaviour is where the invocation families diverge and where most data-loss and duplicate-processing bugs live. Synchronous invocations are not retried by the platform at all -- the error is returned and the caller decides. Asynchronous invocations are retried by the platform, twice by default on Lambda, with backoff between attempts, and the event is discarded afterwards unless a failure destination is configured. Poll-based sources follow the source's own rules: a queue redelivers after the visibility timeout expires and gives up according to the queue's redrive policy, while a stream shard will retry the same batch and, by default, block that shard until the batch succeeds or ages out -- which is how one poison record halts an entire partition.
Because every one of those paths can deliver the same event twice, idempotency is a requirement, not a refinement. Give each event a stable key -- a message ID, an object version, an upstream transaction ID -- record completion against that key in a store with a conditional write, and make the handler return the recorded result on a repeat. Timers on those records matter: too short and a delayed retry re-executes, too long and legitimate reprocessing is blocked.
Dead-letter queues and failure destinations
After retries are exhausted the event has to go somewhere, and 'nowhere' is the default in more places than people expect. A dead-letter queue is the parking lot: a queue or topic that receives events the system could not process, with enough context attached to diagnose and replay them.
The important detail is whose DLQ applies. For asynchronous invocations the function's own failure destination or DLQ catches the event. For queue-based sources the function has no say -- the queue's redrive policy decides, after the configured number of receive attempts, and a DLQ configured on the function is simply not consulted on that path. Teams routinely configure the wrong one, see nothing arrive, and conclude the failures are not happening. For stream sources the equivalent is an on-failure destination plus bisect-on-error, which splits a failing batch to isolate the bad record instead of stalling the shard.
A DLQ with nothing watching it is a slower form of data loss, so treat it as an operational object: alarm on depth greater than zero, keep the original payload and the error together, and build the replay path before you need it. Replay must be idempotent for the same reason retries must be -- most replays re-deliver something that partially succeeded.
Observability when there is no host to log into
You cannot attach a debugger to something that exists for 200 milliseconds, so telemetry has to be designed in. Three signals carry most of the weight. Structured logs with the request ID on every line, because reconstructing a distributed failure from unstructured text across thousands of short executions is hopeless. Traces that propagate context across the asynchronous hops -- the hard part, since a queue breaks the parent-child relationship unless the trace ID rides in message attributes. Metrics chosen for this execution model: invocations, errors, throttles, duration percentiles, concurrent executions against the limit, cold-start count and init duration, and per-source lag such as queue age or stream iterator age.
Two of those deserve alarms even in a small system. Throttles mean the concurrency ceiling is binding and requests are being rejected. Iterator age or queue age means consumption has fallen behind production, which is the earliest reliable signal of a poison record or a slowed dependency -- usually visible well before error rate moves.
The operational surface — networking, identity, secrets, versions
Networking. A function placed in a private network reaches internal resources but loses default internet egress, which needs a NAT path -- a recurring surprise when the function also calls a public API. Private networking also imposes network-interface plumbing that historically dominated cold-start time; that penalty is largely gone on current platforms, but the egress and IP-exhaustion considerations remain.
Identity. Each function should carry its own execution role scoped to exactly what it touches. The anti-pattern is one shared role with broad permissions across dozens of functions, which turns a single injection bug into account-wide access. Per-function roles are the cheapest blast-radius control available and cost nothing at runtime.
Secrets. Environment variables are convenient and are visible to anyone who can read the function configuration. Pull secrets from a secret manager or parameter store at init, cache them in the environment for its lifetime, and handle rotation by treating a credential failure as a signal to refresh rather than by assuming the cached value is permanently valid.
Versions and aliases. Publish immutable versions and point an alias at them so traffic shifting is a configuration change rather than a redeploy. Weighted aliases give canary releases with automatic rollback on an error-rate alarm, which matters more here than in a long-lived service, because a bad serverless deploy reaches full production concurrency within seconds of the first event.
State has to live somewhere, and it is not the function
Handlers are stateless in the sense that matters: nothing you write to local memory or to the writable temporary directory is guaranteed to be visible to the next invocation, and it is guaranteed not to be visible to a concurrent one. Environments are recycled on a schedule you do not control, and a burst creates fresh ones that share nothing.
That said, the environment does persist between invocations, and using it correctly is one of the largest available performance wins. Anything expensive and safe to share -- an HTTP client with a warmed connection pool, a database driver, a parsed configuration blob, a compiled regular expression, a loaded model -- belongs at module scope so it is built once per environment. The discipline is to treat that scope as a cache, never as a source of truth: correctness must not depend on a hit, and a cached value must carry a time-to-live so a rotated credential or a changed feature flag propagates without a deploy.
Genuine state goes to a service. A key-value store for session and idempotency records, an object store for payloads too large to pass through an event body, a queue for work in flight, an orchestrator for multi-step workflows that need to survive a process disappearing mid-way. The temporary directory is useful as scratch space for a single invocation -- unpacking an archive, buffering a download -- and is bounded in size, so a function that streams large objects should stream rather than materialise. The rule of thumb that holds up: if losing it would be a bug, it is not allowed to be in the function.
Where serverless is the wrong shape
Serverless is an excellent fit for event-driven glue, spiky APIs, scheduled jobs, stream processing with modest per-record work, and anything whose duty cycle is low enough that paying for idle servers is absurd. It is a poor fit in a few identifiable cases, and recognising them early saves a rewrite.
Sustained high throughput inverts the cost argument, as the billing section showed. Long-running work collides with the timeout ceiling; the workaround is to decompose into a state machine with checkpointing, which is often the right design anyway but is real engineering, not a configuration flag. Latency floors in the single-digit milliseconds are hard when any invocation can be cold. Heavy shared state or large in-memory caches fight the model directly, since every environment holds its own copy and environments come and go. Chatty synchronous call chains -- function calling function calling function -- pay concurrency and latency at every hop and are better expressed as an orchestrator or an event chain.
The honest summary is that serverless moves work rather than removing it. You stop operating hosts and start operating concurrency limits, retry semantics, idempotency keys and dead-letter queues. That trade is usually worth making. It is not the same as having no operations.
The same model wearing three vendor badges
The vocabulary differs; the architecture does not. AWS Lambda pairs with API Gateway, EventBridge, SQS and Kinesis, and expresses poll-based sources as event source mappings. Azure Functions folds the source into the code through input and output bindings, and its hosting plans -- Consumption, Premium, Dedicated -- are essentially a dial between scale-to-zero and always-warm, with Premium existing precisely to buy away cold starts. Google's Cloud Functions now sit on Cloud Run, which makes the underlying container explicit and permits a single instance to serve multiple concurrent requests -- a genuine model difference, since it decouples instance count from request count and changes how you reason about shared in-process state.
Beyond the big three, edge runtimes -- Cloudflare Workers and similar -- use isolates rather than containers, which drops cold start to near zero at the cost of a restricted runtime and tight CPU-time budgets. Kubernetes-native options such as Knative give the same scale-to-zero-and-back behaviour on your own cluster.
When you compare offerings, compare on the four things this article has been about and ignore the branding: how concurrency is allocated and limited, what the cold-start floor is and what it costs to remove, how retries and dead-lettering work per source, and what the timeout ceiling is. Everything else is portable with modest effort. Those four decide whether your design survives the move.