Secure retrieval: permissions, caches, and citations

What this experiment demonstrates

Five synthetic documents feed a small retrieval pipeline. Each has live readers in the source system and a copy of those readers in the search index. Earlier, Ana from sales asked both questions while the pricing exception memo was still shared with sales by mistake, and the answers were cached. The default state is just after the owner revoked that mistake.

The page does not run a vector database or a language model. Similarity scores are fixed fixture values with a relevance floor of 0.5, and an answer is the list of claims taken from the returned chunks. That makes every difference traceable to one control: where the permission filter runs, whether permission changes have propagated, how the cache key is formed, whether chunks are rechecked against the live ACL, and whether citations are revalidated for the person viewing them.

Run controlled comparisons

Start with the default. The cache key is the query text only, so the pre-revocation answer is reused. Citation revalidation withholds the memo claim, which is why the unauthorized-claims metric stays at zero while stale cache reuse is one. Turn revalidation off to see the leak that display checks were hiding. Then change the cache key to include groups and cited epochs: the old entry becomes unreachable and the answer is regenerated without the memo.

Switch the viewer to Lee, the contractor, and set the cache key back to query text only. Lee receives the answer cached for Ana, a cross-user reuse. Next disable the cache and compare filtering modes. With top-k first and filtering afterwards, the three nearest chunks are all unreadable for contractors, so the answer is empty. Filtering inside the query returns the published price list. Choose no permission filter with the live recheck off to see unauthorized text enter model context, even though display revalidation still hides it.

Finally select nightly sync. Grants and revocations do not reach the index until the sync runs, and an edited returns policy is still served at version 42. The live recheck and citation revalidation are safety nets for that delay, not replacements for propagation.

Inspect the trace and keep evidence

The permissions table compares live readers with the readers recorded in the index and highlights rows that have drifted. The answer panel shows each claim, the cited source and version, and its state for this viewer. Start trace hides the decision steps, Next step reveals them one at a time, and Show all or the slider returns to the full sequence. Metrics always describe the complete run. Reset restores the default revocation scenario, and Export JSON downloads the inputs and outcome so two runs can be compared.

Limits and production translation

The fixture has one cached entry per question, perfect group expansion, and exact version numbers. It does not model approximate-index recall, embedding drift, cache eviction, event delivery failures, or timing. Real systems need tests against the actual store: row-level security that applies to the service role, filter semantics for missing and negated values, cache keys that include permission state, and canary documents that measure how quickly a revocation reaches every derived store. Use synthetic principals and documents for those tests, never real customer data.