Evidence label: deterministic synthetic model. This lab runs a small JavaScript model of access-controlled vector search. It generates 1,600 synthetic chunk vectors in 20 topics, assigns each to one of 50 access groups, builds a neighbour graph, and serves synthetic users who belong to 30, 5 or 1 groups. Each query is answered with the selected strategy on that graph, and recall is measured against an exact search over the chunks the user may see right now. Latency is extrapolated to a 5 million chunk index with simple cost assumptions. No vector database or identity system is contacted.
Play the lab
How to play
Pick a strategy: post-filter, filtered graph search, exact scan of the allowed set, or a planner that scans small allowed sets and walks the graph otherwise. Adjust oversampling for the post-filter, the beam width for the graph and the planner threshold. Choose nightly or event-driven ACL sync and whether to re-check final results against the live ACL. The tiles show overall recall, the worst user class, results returned, latency and revoked documents returned. Events add cross-team questions or a wave of revocations. Missions load a broken setup; three stars go to the leaner or safer configuration.
What the model assumes
Chunks are unit vectors around 20 topic centres; 80% of chunks belong to a group aligned with their topic. The graph links each chunk to its 10 nearest neighbours in its topic plus two random long links, and beam search starts from four fixed entry points. Filtered graph search uses hnswlib-style semantics: disallowed nodes are traversed but not returned, and the search stops only when the result set is full. Latency assumes 1.5 ms overhead, 0.004 ms per graph distance computation (scaled up for searches that exhaust the graph), 0.00004 ms per scanned vector across a 5 million chunk index, and 4 ms for a live ACL re-check. Revocations remove one group from half the users; with nightly sync the index still lists it.
What it cannot tell you
Real ACLs have many groups per document, nested groups and deny rules, and real graphs are multi-layer HNSW with very different connectivity. Engine planners use their own cost estimates and thresholds. The latency extrapolation is a rough illustration, not a benchmark. Use the lab to understand why selectivity and correlation drive the choice of strategy, then measure your engine with your own principal distribution.
Read the full explanation
- Permission-Aware RAG: Filtering by Access Control Without Destroying Recall (companion article)
- Keeping RAG Indexes Fresh: Incremental Updates, Deletes and Tombstones
- HNSW Tuning for RAG: M, efSearch and the Recall Versus Latency Trade
All AI in Production labs · Lab catalog
- ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data (SIGMOD 2024)
- Qdrant documentation: Indexing (payload index and filterable HNSW)
- pgvector README (filtering and iterative index scans)
- Elasticsearch reference: k-nearest neighbor (kNN) search
- hnswlib: Header-only C++/Python library for fast approximate nearest neighbors
- Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)
Views are my own and do not represent any employer, past or present.