Load balancing under uneven service times

Recent requests

Compare distribution with actual work in flight

Four fictional backends receive generated requests. Each admitted request occupies one slot for its configured service duration and then succeeds or fails. A request sent to a backend whose slots are full is rejected immediately, with no retry or hidden queue. Backend cards show routed, admitted, completed, failed and rejected counts separately. At every snapshot, generated equals succeeded plus failed plus rejected plus in-flight work. Resetting rewinds this deterministic history; no old completion timer can decrement the new run’s counters.

Choose a routing policy

Round robin rotates through eligible backend IDs. Least active selects the eligible backend with the fewest occupied slots and rotates ties. Smooth weighted round robin accumulates per-backend scores, chooses the highest score and subtracts the eligible total weight from the winner; ties favor the first backend ID. Seeded random chooses uniformly from eligible backends, so it is repeatable but need not produce equal totals in a finite run. A policy chooses once; a full chosen backend causes rejection rather than an automatic retry on another backend. Weight is used only by the weighted policy.

Set workload, capacity and backend differences

Requests per second controls integer arrivals from zero to forty. Slots per backend sets the same concurrency limit for all four nodes. Normal service duration applies to B0, B1 and B3; B2 service duration lets one node be slower. Duration inputs are in tenths of a second: five means half a second and twenty means two seconds. B2 routing weight ranges from one to eight, while the other nodes have weight one. Random seed changes random-policy selections; deterministic policies do not use it. Every scenario edit rebuilds the trace and pauses at zero.

Measure a known distribution before introducing overload

Disable the scheduled outage, set both service durations to one tick and request rate to four. Round robin sends sixty of the two hundred forty requests to each backend over the minute. Change rate to five, policy to weighted, and B2 weight to two. The three ordinary nodes receive sixty each and B2 receives one hundred twenty, matching a one-one-two-one share. These are routed request counts, not measured throughput capacity. Now restore defaults, disable the outage and compare round robin with least active: the slower B2 accumulates active work, and least active sends it fewer new requests in this workload.

Separate a health decision from a successful response

Scheduled B2 outage marks that node unhealthy from twenty seconds inclusive until forty seconds exclusive. Exclude unhealthy backends uses that exact fixture state to remove B2 from future routing during the outage. Disable exclusion to continue selecting it: requests admitted while B2 is down fail after their service duration. Their outcome is fixed at admission, so recovery does not rescue an already failing request and an earlier healthy admission can still succeed during the outage. This is a chosen simplification, not a model of transport failure detection. Weighted scores are retained while a backend is excluded and resume when it becomes eligible again.

Use the timeline to inspect overload and recovery

Play runs sixty modeled seconds at normal pace. Pause freezes the selected counters and occupied slots. Next second advances one second, Restart rewinds the same scenario, and Show final state jumps to the minute boundary. Speed offers half, normal and double pace; Inspect time scrubs both ways. Restore defaults resets all scenario fields. The recent-request ledger shows the latest twelve generated records and their outcome at the inspected time. Exact evidence includes backend counters and an in-flight sample. Work still active at sixty seconds is not falsely counted as completed.

Keep the experiment’s scope explicit

This is a request-routing and concurrency model, not a production load balancer, HTTP connection pool or network benchmark. The service budget is per backend, requests do not share work capacity, and the model has no waiting queue, autoscaler, circuit breaker, stickiness, client retry or measured percentile latency. Health information is immediate and perfect when enabled, rather than obtained by periodic probes. Completions happen before new admissions in each one-hundred-millisecond step. Fixed service durations and a single scheduled outage make comparisons reproducible; real latency distributions, probe delays and correlated failures require additional modeling.