Evidence label: deterministic synthetic model. This lab runs a small JavaScript model of a RAG index over one synthetic day. It tracks 20,000 synthetic documents and their chunks, generates creates, updates and deletes minute by minute, passes them through either a nightly rebuild or an incremental pipeline with micro-batches and a fixed number of embedding workers, and keeps per-document index state: version, chunk count, orphan chunks and tombstones. Compaction, query outcomes and search latency follow simple stated rules. No database, embedding model or search engine is involved.
Play the lab
How to play
Choose nightly rebuilds or incremental change capture, the batch interval, the number of embedding workers, how deletes are handled, the chunk ID scheme and the compaction threshold. Each change replays the same synthetic day with the same random seed. The tiles show the share of answers from current content, stale or missing answers, deleted documents served per thousand queries, p95 lag, p95 search latency and the daily cost of embedding workers. Events add a bulk re-import, a takedown of 1,600 documents and users who mostly ask about recent changes. Each mission loads a broken pipeline; three stars reward the tighter configuration.
What the model assumes
Changes arrive as Poisson streams of 2 creates, 3.5 updates and 0.5 deletes per minute. Updates change a document's chunk count by minus three to plus two. Each worker embeds 200 chunks per minute and costs an illustrative $1.20 an hour. Positional IDs overwrite chunks in place and leave extra old chunks as orphans; doc-scoped IDs tombstone every old chunk. Ignored deletes stay searchable until a rebuild. Search latency is 20 ms scaled by 1 + 2.2 times the tombstone share, and 60% slower while compaction runs; compaction takes time proportional to the index size. A query about a document with orphans quotes the old text 35% of the time, and tombstones reduce the chance that a current document is found.
What it cannot tell you
Real engines differ in how they store deletes, how merges are scheduled and how expensive refreshes are, and real update streams are burstier and correlated with user questions in ways this model only imitates. Latency and cost constants are illustrative. Use the lab to see how batch interval, worker capacity, ID scheme and compaction interact, then measure lag and tombstone growth in your own pipeline.
Read the full explanation
- Keeping RAG Indexes Fresh: Incremental Updates, Deletes and Tombstones (companion article)
- Permission-Aware RAG: Filtering by Access Control Without Destroying Recall
- Chunking Strategies Compared: Fixed, Recursive, Semantic and Late Chunking
All AI in Production labs · Lab catalog
- Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs (HNSW)
- FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search (2021)
- Qdrant documentation: Optimizer (vacuum and deleted_threshold)
- hnswlib README (mark_deleted and allow_replace_deleted)
- Elasticsearch reference: Merge settings
- pgvector README (HNSW indexing and vacuuming)
Views are my own and do not represent any employer, past or present.