Why it matters

Cache key design shapes correctness. Understanding shapes cache safety.

Advertisement

The architecture

Hash: prompt + model + temperature + top-p + tools + tenant + version.

SHA256 stable digest.

Cache key compositionPrompt + paramsuser + model + tempTools + tenantand versionSHA256 keystable digestMissing any input = wrong hits; include model + version + tenant
Cache key design.
Advertisement

How it works end to end

Serialize deterministically (sorted JSON).

Include model + version.

Tenant / user isolation.

Tool set signature.