Evidence label: deterministic synthetic model. This lab runs a small JavaScript model of query rewriting for retrieval. It generates a synthetic catalogue of 480 passage vectors (120 items, four aspects each, grouped into topics with similar siblings) and 200 synthetic queries of four types, then simulates a rewriter LLM that condenses follow-ups, writes paraphrases or writes hypothetical passages, with latency, token cost and knowledge limits drawn from simple assumptions. Retrieval is exact cosine search over the synthetic vectors with reciprocal rank fusion. No language model, embedding model or search engine is called.
Play the lab
How to play
Turn condensing on or off, choose an expansion (none, multi-query with RRF, or HyDE), the number of variants, whether the original query is kept, whether exact IDs bypass the rewriter, the rewriter model and the timeout. Every change replays the same 200 queries, so the comparison with the grey no-rewrite bars is fair. Watch the harmed-queries tile as closely as the hit rate. Events add a surge of chat follow-ups, a batch of new items the rewriter does not know, or a threefold slowdown of the rewriter. Each mission loads a broken setup; three stars go to the cheapest or fastest configuration that meets the goal.
What the model assumes
Each passage vector mixes a topic, an item and an aspect. Vague queries carry weak item and aspect signals plus noise; follow-ups omit the item entirely; ID queries carry a strong item signal; well-formed queries sit close to their passage. The small rewriter recognises 85% of known items and the large one 95%; ID strings are recognised 20% of the time and paraphrases drop them 60% of the time. Unknown items are replaced in HyDE passages by a sibling from the same topic. Rewrite latency is time-to-first-token plus output tokens times a per-token time, with a log-normal spread and a small heavy tail; prices are illustrative per-million-token rates.
What it cannot tell you
Real queries, embeddings and rewriter behaviour are far richer than this vector mixture, and real hybrid search would rescue many identifier queries that the dense-only model here loses. The numbers show the direction and shape of the trade-offs (who benefits, who is harmed, what the latency tail does), not the gains you will see. Measure on a labelled sample of your own traffic, per query type, before shipping a rewrite step.
Read the full explanation
- Query Rewriting and HyDE: Fixing the Question Before Retrieval Runs (companion article)
- Reciprocal Rank Fusion Tuning: When BM25 Rescues Dense Retrieval Misses
- Multi-Hop Questions in RAG: Iterative Retrieval and Query Decomposition
All AI in Production labs · Lab catalog
- Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE, ACL 2023)
- Query2doc: Query Expansion with Large Language Models (EMNLP 2023)
- Query Rewriting for Retrieval-Augmented Large Language Models (EMNLP 2023)
- Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models (2023)
- Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods (SIGIR 2009)
Views are my own and do not represent any employer, past or present.