Why it matters

Public LLM APIs face abuse. Ingress control shapes safety + cost.

Advertisement

The architecture

Auth: token per user/tenant.

Rate limit: per token.

WAF: standard + LLM-specific rules.

Ingress control stackAuthper-tenant tokenRate limitreq + tokens per minWAFinjection rulesLLM-specific: rate limit by tokens not just requests; block prompt patterns
Ingress.
Advertisement

How it works end to end

Rate limit by tokens + requests.

WAF: block known injection patterns.

Circuit breaker: on abuse.

DDoS: CloudFront + WAF + Shield.