Problem
metric{user_id=X} creates one time series per user. Million users = million time series per metric. Explodes.
Advertisement
Rules
Never label with user_id, request_id, session_id, IP. Anything unbounded. Bucketize into categories.
Advertisement
Good labels
Low-cardinality: environment (dev/prod), region, service, endpoint, HTTP method, status_code (5xx bucket). Bounded set.