Why architecture matters here
RAG fails on chunking (too big or small), retrieval (missed context), and hallucination (model ignores context). Architecture matters because each layer decides quality independently.
The architecture: every piece explained
The top strip is the ingest pipeline. Documents raw. Chunker semantic splits. Embedder vectorizes. Vector store holds embeddings + metadata.
The middle row is the query path. Query embed vectorizes user question. Retrieval ANN + optional hybrid (BM25 + vector). Reranker — cross-encoder — refines top-K. Generator LLM with retrieved context.
The lower rows are quality. Eval harness — faithfulness + relevance + hit rate. Feedback loop from user signals. Ops — freshness, PII, governance.
End-to-end flow
End-to-end: user asks. Query embedded. Retrieval fetches top-20 chunks. Cross-encoder reranks to top-5. LLM generates with citations. Eval harness confirms faithfulness. Feedback tunes chunk size + retrieval params next cycle.