Your RAG index goes stale the week after launch

The notebook demo works. Documents go in, answers come out, the board is pleased. Then the source data changes and the assistant keeps quoting last month’s pricing page.

Retrieval quality is a data engineering problem long before it is a model problem. Three things decide whether a RAG system stays correct.

1. Update in place, not by rebuilding

A full re-embed of the corpus on every change is slow and expensive, so teams stop doing it. Design ingestion to be incremental from day one:

• Store a content hash and a source version next to every chunk.
• On change, re-chunk only the changed document and upsert by a stable chunk id.
• Run the job on change events, not on a nightly schedule that nobody watches.

2. Deletions must remove retrievability

When a document is withdrawn, its chunks have to leave the index. This matters most for regulated data: a retracted clinical note or a revoked contract that still shows up in answers is a compliance incident, not a quality issue. Track which chunk ids belong to which source document so a delete is one query.

3. Someone owns freshness

Add two numbers to your dashboard: the age of the oldest un-refreshed document, and the count of source changes still waiting to be indexed. Alert on them like you would on consumer lag. If nobody is paged when the index falls a day behind, it will fall a month behind.

Access control belongs in the index

If different users may see different documents, store the permission metadata with each chunk and filter at query time. Filtering after retrieval leaks through ranking and wastes context window.

A quick test before launch: change one source document, delete another, and time how long until the answers reflect both. That number is your real freshness SLA.