RAG retrieval workbench

Follow evidence through retrieval

This experiment searches six fictional Atlas runbooks. Query text is lowercased, split into letter and number terms, and filtered through a small explicit stop-word list. The source metadata gates run before the eligible search index is built. Every remaining document receives a computed lexical cosine score, not a hand-assigned relevance number. Inspect a source row or use Document to see its exact text, term contributions and reason for inclusion or exclusion. The answer panel quotes admitted source text with source identifiers; it does not call an LLM.

Use the controls

Query changes the terms used for ranking. Workspace selects the caller's fictional team. Enforce workspace scope prevents other teams from entering retrieval; disabling it intentionally demonstrates a missing access filter using harmless example data. Maximum source age excludes old documents. Top K caps how many positive-scoring documents proceed to context packing. Minimum cosine score rejects weak matches. Context source-word budget admits whole documents in ranked order when they fit the remaining allowance, skipping oversized documents without truncating them. Restore defaults resets those policies. Play reveals the six-stage computation over sixty seconds, Pause freezes it, Next stage advances inspection, Restart rewinds, Speed changes teaching pace, and Inspection seconds scrubs the same result. Editing an input resets playback without autoplay.

Try a failure boundary

Start with the backup query and inspect the current Alpha schedule. Increase maximum age to 365 days, top K to six and budget to 120 words: the archived schedule can now appear beside newer evidence. A citation does not automatically resolve contradictory sources. Disable workspace scope and inspect whether the Beta schedule enters context; relevance is not permission. Set the budget to zero: the responder must abstain even when retrieval finds matches. Try a query containing only unknown words and verify that there is no invented answer. Then restore defaults and query deployment rollback to watch an entirely different source rise in the ranking.

Understand the model limits

Term weight is (1 + natural log of term count) multiplied by a smoothed inverse-document frequency, log((eligible document count + 1)/(document frequency + 1)) + 1. Cosine divides the query-document dot product by their vector lengths. This is sparse lexical retrieval, not a neural embedding, semantic reranker or vector-database benchmark. Filtering changes the eligible corpus and therefore its inverse-document frequencies. Empty and zero-overlap queries produce no candidates. The budget counts whitespace-separated source words only, excluding prompt instructions, identifiers and the query; it is not an LLM tokenizer or full context-window limit. Quoted evidence can still be stale, irrelevant or contradictory. No model generation, external document retrieval, real secrets or production access control runs here.

Background: query vectors and cosine ranking. Continue with the AICassindra grounding article and retrieval evaluation metrics.