Why it matters
Every LLM deployment that touched real user data must consider PII leakage. Regulators are increasingly focused on this. Mitigation is complex but necessary.
The architecture
Memorization: LLMs assign high probability to training sequences seen many times or verbatim. Extract by prompting with prefixes that lead to memorized completions.
Extraction attacks: adversary crafts queries that elicit memorized text. Studies show billions of tokens extractable from public models.
How it works end to end
Mitigations: deduplication of training data reduces repetition-based memorization. Differential privacy during training adds noise that bounds memorization mathematically.
Output filtering: post-hoc detect and redact PII patterns in output. Doesn't prevent all leaks but catches common ones.
Access control: use retrieval instead of training. If PII is in a retrieval index, access can be controlled per-query.