Begin with the problem
A correct answer can become stale tomorrow. Versioning and freshness controls decide which document is current and remove obsolete evidence from search.
observe failure → locate pipeline stage → change one component → evaluateWhat you will learn
- Explain Document Versioning & Freshness in simple language before using its technical details.
- Follow the mechanism step by step through a small RAG example.
- Connect this topic to the modules before and after it.
- Decide when to use it, when not to use it, and what to measure in production.
Current real-system grounding: OpenAI’s evaluation guidance supports testing changes against datasets rather than trusting a few demos. RAG needs separate retrieval and answer checks.
The product example proves that the pattern is used in a real system. It does not mean every provider uses the same hidden algorithm, defaults, limits, or pricing.
1. The problem this module solves
Module 25 covered detecting and resolving conflicts once they exist. This module addresses the really better question: how do you prevent stale, conflicting information from accumulating in your knowledge base in the first place? Recall Module 4’s offline/online split — this module goes deeper into what “keeping the offline index current” really requires.
2. The Core Problem
Document changes (a policy gets updated).
But the VECTOR INDEX (Module 12) still contains embeddings from the
OLD, now-outdated version -- because indexing (Module 4's offline
phase) doesn't happen automatically just because a source document
changed somewhere.Document update
↓
Re-ingestion (Module 5)
↓
Re-chunk (Modules 7-9)
↓
Re-embed (Module 10)
↓
Update the index (Module 12)Every one of these steps needs to really happen again for a single updated document — this isn’t automatic, and a system that doesn’t actively trigger this pipeline on document changes will silently serve stale information indefinitely.
3. Detecting Staleness — Comparing Timestamps
Document's LAST MODIFIED timestamp (captured at ingestion, Module 5)
vs.
The INDEX's record of when THIS document was LAST indexed
If document.last_modified > index.last_indexed:
This document is REALLY STALE -- it needs re-indexingThis directly depends on Module 5’s metadata capture — without a
real last_modified timestamp captured at ingestion, there’s no
principled way to even detect that a document has changed at all.
4. Incremental vs. Full Reindexing
FULL REINDEXING: re-process the ENTIRE knowledge base from
scratch -- really simple, but wasteful and
slow for large knowledge bases where only a
FEW documents actually changed
INCREMENTAL REINDEXING: only re-process documents
INDEXING: that REALLY changed since the
last indexing run -- more complex to
implement, but really efficient at
real scaleFor a small knowledge base, full reindexing on a schedule (e.g., nightly) is often really simple and sufficient. For a large, frequently-changing knowledge base, incremental indexing — using exactly Section 3’s timestamp comparison — becomes really necessary to keep the index current without wasting substantial compute reprocessing documents that never actually changed.
5. Archiving Old Versions — Not Just Deleting
When a document is UPDATED, what happens to the OLD version?
Option 1 -- DELETE the old version entirely: loses historical
record; may
really be
needed for audit
or compliance
purposes
Option 2 -- ARCHIVE the old version, marked as
SUPERSEDED (metadata flag), but really
KEPT in a separate, non-default- searched
index: preserves
history while
preventing it
from being
retrieved
during normal
searchThis directly connects back to Module 25’s conflict problem — the reason Policy 2024 and Policy 2026 both showed up as candidates was precisely because the old version was never properly archived or flagged as superseded. Proper versioning PREVENTS this class of conflict from arising in the first place, rather than needing to resolve it after the fact.
6. A Real Developer Example
TechCorp implements a document versioning strategy for their HR
policy knowledge base:
1. Every document carries a `version` and `effective_date` field
(Module 9's metadata, extended)
2. When HR uploads an updated travel policy, the SYSTEM automatically:
- Marks the PREVIOUS version's chunks with status="superseded"
- Excludes superseded chunks from DEFAULT retrieval (Module 15's
metadata filtering, applied here directly)
- Re-indexes the NEW version's chunks as the active,
status="current" version
Now, when an employee asks about the hotel limit, retrieval ONLY
considers status="current" chunks -- Module 25's conflict scenario
literally CANNOT occur, because the outdated version is structurally
EXCLUDED from search, not just deprioritized.7. A Simple Agentic AI Connection
An agent’s knowledge-base search tool should really respect versioning metadata by default — always filtering to current, active documents unless a user really asks about historical policy (in which case the agent could deliberately search the archived, superseded index instead) — directly connecting Module 15’s filtering mechanism to this module’s versioning strategy.
8. How Is This Used in AI?
🤖 How Is This Used in AI?
Production RAG systems managing really evolving knowledge bases implement automated re-indexing triggers (on document upload or scheduled checks) and explicit version/status metadata — precisely to prevent the stale-answer and conflicting-document problems from Module 25, by keeping the searchable index really synchronized with the actual, current source of truth.
9. Real-World Applications
- HR and policy knowledge bases with periodically updated documents
- Legal and compliance systems requiring clear document version history
- Any enterprise RAG system where source documents really change over time
10. Common Mistakes
Incorrect idea: Indexing a knowledge base once and never re-indexing it.
Why it is incorrect: As shown directly in Section 2, this really causes the index to silently drift out of sync with real, current source documents.
Incorrect idea: Deleting old document versions instead of archiving them.
Why it is incorrect: As shown directly in Section 5, this loses potentially really important historical record for audit or compliance purposes.
Incorrect idea: Not structurally excluding superseded versions from default retrieval.
Why it is incorrect: As shown directly in Section 6, this is precisely what allows Module 25’s conflict scenario to occur in the first place.
11. Limitations
- Detecting staleness really depends on accurate
last_modifiedmetadata being captured and maintained at the source — an unreliable source system undermines this entire mechanism - Incremental reindexing adds real implementation complexity compared to simple, scheduled full reindexing — a real trade-off requiring real scale to justify
12. Quick Reference — The Whole Idea in One Diagram
Document last_modified > index last_indexed?
YES -> STALE -> trigger re-ingestion, re-chunk, re-embed,
re-index (Module 4's offline pipeline, re-run)
Document UPDATED:
Old version -> marked SUPERSEDED, archived (excluded from
default search)
New version -> marked CURRENT, actively indexed
Default retrieval: ONLY searches status="current" documents --
Module 25's conflicts structurally prevented13. Code — Implementing Freshness Detection and Version Management
🎯 Target of this example: implement Section 3’s staleness detection and Section 6’s real developer example directly — detecting which documents need reindexing, and structurally filtering search to only current, non-superseded chunks.
Example 1 — Simple
from datetime import datetime
def needs_reindexing(document_last_modified: str, index_last_updated: str) -> bool:
"""Directly implements Section 3's staleness detection --
comparing document modification time against when the index was
last updated for that document."""
doc_time = datetime.fromisoformat(document_last_modified)
index_time = datetime.fromisoformat(index_last_updated)
return doc_time > index_time
documents = {
"travel_policy": {"last_modified": "2026-03-15T10:00:00", "last_indexed": "2026-01-01T00:00:00"},
"facilities_faq": {"last_modified": "2025-06-01T00:00:00", "last_indexed": "2026-01-01T00:00:00"},
}
for name, doc in documents.items():
stale = needs_reindexing(doc["last_modified"], doc["last_indexed"])
print(f"{name}: {'NEEDS RE-INDEXING' if stale else 'up to date'}")Expected Output:
travel_policy: NEEDS RE-INDEXING
facilities_faq: up to dateWhat we conclude from this example: travel_policy was modified
AFTER it was last indexed, correctly flagging it as stale. facilities_faq
hasn’t changed since its last indexing, correctly identified as
up to date — exactly Section 3’s timestamp comparison, made directly
operational.
Example 2 — Intermediate
from dataclasses import dataclass
from enum import Enum
class DocumentStatus(Enum):
CURRENT = "current"
SUPERSEDED = "superseded"
@dataclass
class VersionedChunk:
text: str
document_id: str
version: str
status: DocumentStatus
def search_current_only(query_relevant_chunks: list) -> list:
"""Directly implements Section 6's real developer example --
structurally EXCLUDES superseded chunks from default retrieval,
preventing Module 25's conflict scenario entirely."""
return [c for c in query_relevant_chunks if c.status == DocumentStatus.CURRENT]
chunks = [
VersionedChunk("Hotel limit is $8000/night.", "travel_policy", "2024", DocumentStatus.SUPERSEDED),
VersionedChunk("Hotel limit is $10000/night.", "travel_policy", "2026", DocumentStatus.CURRENT),
]
default_search_results = search_current_only(chunks)
print(f"Total chunks in knowledge base: {len(chunks)}")
print(f"Chunks returned by DEFAULT search: {len(default_search_results)}")
for chunk in default_search_results:
print(f" [{chunk.version}, {chunk.status.value}] {chunk.text}")Expected Output:
Total chunks in knowledge base: 2
Chunks returned by DEFAULT search: 1
[2026, current] Hotel limit is $10000/night.What we conclude from this example: the superseded 2024 chunk is structurally excluded from default search results, even though it’s still stored in the knowledge base (for historical/audit purposes, Section 5) — only the current, 2026 chunk is returned. This directly prevents Module 25’s conflict scenario from ever occurring during normal retrieval, rather than requiring conflict detection after the fact.
Example 3 — Production Grade
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
class DocumentStatus(Enum):
CURRENT = "current"
SUPERSEDED = "superseded"
@dataclass
class VersionedDocument:
document_id: str
version: str
status: DocumentStatus
effective_date: str
superseded_by: str = None
class DocumentVersionManager:
"""A production-style version manager IMPLEMENTING Section 6's
full workflow -- when a NEW version is published, the PREVIOUS
version is AUTOMATICALLY marked superseded, structurally
preventing Module 25-style conflicts from ever accumulating."""
def __init__(self):
self.documents: dict = {} # document_id -> VersionedDocument
def publish_new_version(self, document_id: str, new_version: str, effective_date: str) -> dict:
"""Publishing a new version AUTOMATICALLY supersedes any
existing current version of the SAME document."""
actions_taken = []
# Find and supersede any EXISTING current version of this document
for doc_key, doc in list(self.documents.items()):
if doc.document_id == document_id and doc.status == DocumentStatus.CURRENT:
doc.status = DocumentStatus.SUPERSEDED
doc.superseded_by = new_version
actions_taken.append(f"Superseded version {doc.version}")
# Register the NEW version as current
new_key = f"{document_id}_v{new_version}"
self.documents[new_key] = VersionedDocument(
document_id=document_id, version=new_version,
status=DocumentStatus.CURRENT, effective_date=effective_date,
)
actions_taken.append(f"Published version {new_version} as CURRENT")
return {"document_id": document_id, "actions": actions_taken}
def get_current_version(self, document_id: str) -> VersionedDocument:
for doc in self.documents.values():
if doc.document_id == document_id and doc.status == DocumentStatus.CURRENT:
return doc
return None
manager = DocumentVersionManager()
result1 = manager.publish_new_version("travel_policy", "2024", "2024-01-01")
print(f"Publishing 2024: {result1['actions']}")
result2 = manager.publish_new_version("travel_policy", "2026", "2026-01-01")
print(f"Publishing 2026: {result2['actions']}")
current = manager.get_current_version("travel_policy")
print(f"\nCurrent version: {current.version} (effective {current.effective_date})")Expected Output:
Publishing 2024: ['Published version 2024 as CURRENT']
Publishing 2026: ['Superseded version 2024', 'Published version 2026
as CURRENT']
Current version: 2026 (effective 2026-01-01)What we conclude from this example: publishing the 2026 version
automatically and correctly supersedes the 2024 version, with the
actions_taken log making this transition really auditable —
exactly the automated version management workflow Section 6 described,
implemented as real, structural logic that prevents conflicting
current versions from ever coexisting in the first place, rather than
relying on manual cleanup or after-the-fact conflict resolution.
14. Interview Questions
Q: Why doesn’t a vector index automatically stay in sync with source documents as they change?
Ans: Indexing (embedding and storing a document’s chunks) is a distinct step from the source document itself changing — updating a policy document doesn’t automatically trigger re-ingestion, re-chunking, and re-embedding. This entire pipeline needs to really run again for a document to be reflected in the index, which is precisely why systems need explicit mechanisms — either scheduled reindexing or change-triggered reindexing — to keep the searchable index synchronized with the actual current state of source documents.
Q: What’s the difference between full reindexing and incremental reindexing, and when might each be appropriate?
Ans: Full reindexing reprocesses the entire knowledge base from scratch, which is simple but potentially wasteful for a large knowledge base where only a small number of documents actually changed. Incremental reindexing only reprocesses documents that have really changed since the last indexing run, comparing modification timestamps against last-indexed timestamps. Full reindexing is often sufficient for smaller knowledge bases on a reasonable schedule, while incremental indexing becomes really necessary at larger scale to avoid wasting substantial compute reprocessing unchanged documents.
Q: Explain how proper document versioning prevents the conflicting- documents problem covered in the previous module, rather than just helping resolve it after the fact.
Ans: When a document is updated, a proper versioning system automatically marks the previous version as superseded and excludes it from default retrieval — only the current, active version remains searchable by default. This means an outdated and a current version of the same policy can never both appear as retrieval candidates simultaneously during normal search, structurally preventing the kind of conflict where two really existing documents disagree, rather than requiring conflict detection and resolution logic to handle it after the fact.
Q: Why might a system choose to archive old document versions rather than simply deleting them?
Ans: Archived, superseded versions can really be needed later for audit trails, compliance requirements, or historical reference — understanding what a policy stated at a specific point in the past can matter for legitimate business or legal reasons. Archiving keeps this historical record available in a separate, non-default-searched location, preserving it without allowing it to interfere with normal, current-focused retrieval — deleting old versions entirely would lose this potentially important historical information permanently.
15. What You Should Remember
- Indexes don’t automatically stay synchronized with source documents — real re-ingestion, re-chunking, re-embedding, and re-indexing must be explicitly triggered, verified directly through a working staleness-detection function.
- Superseded document versions should be archived, not deleted, and structurally excluded from default retrieval — verified directly by showing a superseded chunk correctly absent from search results while still preserved in storage.
- Proper versioning prevents Module 25’s conflicts from arising in the first place — verified directly through a version manager that automatically supersedes old versions when a new one is published.
16. Quick Practice
Design a versioning and freshness strategy for a knowledge base that updates VERY frequently (multiple times per day) versus one that updates RARELY (once a year) — what different reindexing approach would really make sense for each, and why?
17. Next Step
Next: Module 27 — RAG Security & Access Control — closing Level 6: the real security requirements around retrieval, including enforcing permissions and defending against prompt injection via retrieved documents.
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed