Practical LLM Security · RAG and Knowledge Security · Part 2 of 5
Enforce document permissions inside the vector query, avoid filtered-search starvation, and test the database and vector-store behaviors that silently bypass access rules.
From principle to query plan
The rule is easy to state: a user should only retrieve chunks they are allowed to read. The site's earlier guide to ACL-aware retrieval for enterprise RAG covers the principles, including carrying ACLs to chunks and filtering inside the search. This chapter is about the implementation details where correct principles still produce wrong results: how approximate indexes apply filters, how row-level security is bypassed by the role your service uses, how deny-only filters can mean allow-all, and how rerankers and query rewriters become side channels.
The example is Northstar's support assistant again. Chunks carry an allowed_groups array and a tenant identifier. Agents in the returns team may read returns policies and their own region's escalation notes. Contractors may read only published policies.
A concrete failure: filtered search that returns nothing useful, then everything
The first implementation retrieves the 40 nearest chunks from an HNSW index and then filters them by the user's groups in application code. For a contractor, most of the nearest chunks are internal escalation notes, so after filtering only one chunk remains. Answers become poor and the team receives complaints. A developer fixes quality by raising the candidate count to 2,000 and moving the filter into a second service that occasionally times out. Under timeout, the fallback path returns the unfiltered candidates because the code treated an empty permission response as no restriction.
Two separate defects are present. Post-filtering an approximate search starves restricted users of results. The pgvector documentation states this directly: with approximate indexes, filtering is applied after the index is scanned, so if a condition matches 10 percent of rows with the default hnsw.ef_search of 40, only about four rows will match on average. The second defect is a fail-open fallback. Starvation pressure is exactly what encourages engineers to add the shortcuts that leak.
Architecture: enforce in the store, scope the query, verify the result
| Layer | Control | What it prevents |
|---|---|---|
| Identity | Server-derived principal with group snapshot and epoch | Model or client choosing its own groups |
| Query | Permission predicate inside the vector query | Unfiltered candidates leaving the store |
| Store | Row-level security with a non-owner role | Developer queries that forget the predicate |
| Result check | Recheck each returned chunk against current ACL | Stale index permissions |
| Downstream | Reranker, rewriter, and cache keyed by principal | Side channels and cross-user reuse |
The principal is computed by the application from the authenticated session, never from a model argument. Group membership is expanded once per request into a snapshot with an epoch number, so every downstream component sees the same set and logs can show which epoch decided access.
Push the predicate into the store so unauthorized rows never leave it. In PostgreSQL with pgvector, enabling iterative index scans, available from pgvector 0.8.0, lets the index keep scanning until enough rows satisfy the filter or a scan limit such as hnsw.max_scan_tuples is reached. Strict ordering returns results in exact distance order; relaxed ordering can improve recall with slightly out-of-order results. For a small number of large, stable audiences, partial indexes per audience are another documented option.
Implementation: row-level security that actually applies
Illustrative python; adapt the persistence and identity adapters to your service.
RLS_SETUP = """
ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;
ALTER TABLE chunks FORCE ROW LEVEL SECURITY;
CREATE POLICY chunk_read ON chunks FOR SELECT TO retrieval_reader
USING (tenant_id = current_setting('app.tenant_id')
AND allowed_groups && string_to_array(
current_setting('app.groups'), ','));
"""
def search(conn, principal, query_vector, k=8):
with conn.transaction():
conn.execute("SET LOCAL ROLE retrieval_reader")
conn.execute("SELECT set_config('app.tenant_id', %s, true)",
(principal.tenant_id,))
conn.execute("SELECT set_config('app.groups', %s, true)",
(",".join(principal.groups),))
conn.execute("SET LOCAL hnsw.iterative_scan = strict_order")
return conn.execute(
"SELECT id, doc_id, doc_version, text FROM chunks "
"ORDER BY embedding <=> %s LIMIT %s",
(query_vector, k)).fetchall()Three details matter. First, PostgreSQL documents that table owners normally bypass row security, and superusers and roles with BYPASSRLS always do. If the retrieval service connects as the role that created the table, the policy silently does nothing. FORCE ROW LEVEL SECURITY makes the owner subject to policies, and a dedicated reader role without BYPASSRLS is safer still. Second, if no policy exists after RLS is enabled, PostgreSQL applies default deny, which is the correct failure direction. Third, the settings are transaction-local, so a pooled connection cannot carry one user's groups into another user's query.
The SQL string is part of the illustrative design; the set_config calls are standard PostgreSQL. Adapt the driver calls to your client library and test them.
Vendor filters have their own semantics
Managed vector stores implement permissions as metadata filters, and their semantics deserve the same scrutiny as SQL. Vertex AI Vector Search, for example, supports token restricts organized by namespace, including denylist tokens. Its documentation notes that if a query namespace contains only denylisted tokens, all points not explicitly denylisted match, in the same way an empty namespace matches all points. A design that expresses access as "deny group X" therefore grants access to every datapoint that lacks a deny token, including newly ingested documents whose labels were never set. Express permissions as allow tokens, require the namespace on every datapoint at ingestion, and treat a datapoint without the namespace as unadmitted.
The general rule: read your store's filter semantics for empty, missing, and negated values, then write tests for each case. OWASP's 2025 entry on vector and embedding weaknesses lists inadequate or misaligned access controls and cross-context leaks in shared vector databases as primary risks for exactly this reason.
Group expansion and epoch hygiene
Permissions are usually granted to groups, and groups nest. Expanding nested groups at query time on every request is slow and fragile; expanding them only nightly makes revocation slow. A practical middle path is an identity service that maintains each principal's effective group set with an epoch that increases on any membership change, and that pushes changes as events. The retrieval service reads the snapshot with its epoch and fails closed if the snapshot is older than a configured maximum age. Record the epoch in retrieval logs so investigators can tell whether a disclosure came from a stale membership or from a wrong document label.
Tradeoffs and side channels
Strict filtering inside the index can increase latency for users with narrow access, because the scan must visit more candidates. Partial indexes reduce latency but multiply index maintenance. Separate indexes per tenant give the strongest isolation for multi-tenant products at the cost of operational overhead. Choose per data class: per-tenant indexes for customer data, shared indexes with row-level predicates for internal knowledge with many overlapping groups.
Components after retrieval can still leak. A query rewriter that sees the user's history from another tenant, a reranker service that receives unfiltered candidates, and a semantic cache keyed only by query text all reintroduce cross-user exposure. Pass only filtered candidates to rerankers, and key every cache by tenant, principal or group set, and ACL epoch, as the revocation chapter explains.
Worked case and verification
Northstar rebuilds the contractor path with the predicate inside the query, FORCE ROW LEVEL SECURITY, a reader role, and strict iterative scans. The same contractor question now returns eight published-policy chunks instead of one. The timeout fallback is removed: if the group snapshot cannot be computed, the request fails with a clear message and no retrieval occurs.
Verify with a permission matrix rather than a few examples. Generate synthetic principals for each group combination, synthetic documents for each label combination, and assert for every pair that unauthorized chunks are never returned, including under simulated timeouts of the group service. Add tests that connect as the table owner to prove the force setting is active, tests for missing filter namespaces in the vector store, and tests that the reranker never receives an unauthorized chunk. Track recall for restricted users separately; a sharp drop is an early sign that someone will soon be tempted to post-filter again. The companion secure retrieval lab lets you compare pre-filter and post-filter behavior with deterministic data.
Practice and continue
Open the Secure retrieval: permissions, caches, and citations lab to change the assumptions with deterministic synthetic data and inspect each decision. This chapter belongs to the RAG and Knowledge Security group of Practical LLM Security, which builds on the 25-chapter abuse and fraud prevention series.
Sources and implementation scope
Primary references checked October 16, 2026. Framework details follow the linked documentation and, for ADK, the google-adk 2.12.0 package source; pin and test the version your application uses. The Northstar scenarios, policies, figures, and code are original illustrative designs, not measured production results or complete integrations.