CPU-based autoscale

Default. Works for CPU-bound. Set target ~60%. Beware LLM-bound agents that aren't CPU-limited.

Advertisement

Custom metrics

Queue depth, in-flight requests, LLM latency. More precise than CPU.

Advertisement

Predictive autoscale

Some platforms scale based on time patterns. Good for predictable traffic (business hours).