How LangChain Implements Retrieval

You already know why retrieval-augmented generation works — grounding a model’s answer in real, retrieved documents, reducing hallucination, keeping answers current. That’s not the job of this module. This module, and the five that follow it, are entirely about how, concretely, in real LangChain code — turning theory you already hold into components you can actually build with.

The pipeline, mapped

flowchart LR
    A[Documents] --> B[Document Loaders]
    B --> C[Text Splitters]
    C --> D[Embeddings]
    D --> E[Vector Store]
    E --> F[Retriever]
    F --> G[Relevant chunks]
    G --> H[Model]

Each box in this diagram gets its own module, in order: Document Loaders bring raw files into LangChain’s world. Text Splitters break long documents into genuinely searchable pieces. Embeddings turn text into the numerical vectors your earlier course already covered the mathematics of. A Vector Store holds and searches those vectors. A Retriever is the actual interface your application code calls to get relevant chunks back. And finally, those chunks feed into a model, exactly like the RunnablePassthrough.assign() pattern from Module 9 — carrying original input forward while adding computed context alongside it.

Why this module exists on its own, briefly

Recall Module 9’s own honest limitation: its “research-then-summarize chain” example deliberately avoided real retrieval, since you hadn’t learned it yet. Recall also Module 16’s Agent 6 — a genuinely crude, word-matching “knowledge base search,” explicitly flagged as a placeholder for exactly what this phase now delivers properly.

A first, honest look at the whole pipeline in miniature

Before breaking each piece apart across the next five modules, it’s worth seeing the complete shape once, even in a deliberately minimal form.

from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings
from langchain_core.vectorstores import InMemoryVectorStore

docs = [
    Document(page_content="Our return policy allows returns within 30 days of purchase."),
    Document(page_content="Shipping typically takes 3-5 business days."),
    Document(page_content="Gift cards do not expire and cannot be redeemed for cash."),
]

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = InMemoryVectorStore(embeddings)
vector_store.add_documents(docs)

retriever = vector_store.as_retriever(search_kwargs={"k": 1})
results = retriever.invoke("How long do I have to return something?")

for doc in results:
    print(doc.page_content)

Notice retriever.invoke(...) — the exact same .invoke() interface from every Runnable you’ve used since Module 8. A retriever genuinely is a Runnable, which is precisely why it slots directly into |-composed chains, exactly like a prompt or a model.

The honest difference between this and Module 16’s placeholder

Recall Module 16’s search_knowledge_base — matching by raw word overlap, a genuinely crude approximation. This example instead uses real embeddings, turning each document and each query into a vector, then finding documents whose vectors are genuinely close in meaning — recall from your earlier course exactly why this captures semantic similarity, not just shared words. "How long do I have to return something?" correctly retrieves the return policy document, despite sharing almost no exact words with it at all. That’s the real, concrete difference retrieval makes over the placeholder you built earlier.

Common mistakes worth avoiding

Assuming retrieval alone guarantees a correct answer. A retriever finding the most similar chunk to a query doesn’t guarantee that chunk actually answers the question — recall this exact gap will matter directly once Module 26 covers real RAG prompt design, where the model still needs clear instructions about what to do with imperfect context.

Skipping straight to a real vector database before understanding the basic mechanics. InMemoryVectorStore, used in this module’s example, is genuinely the right starting point for learning — the concepts (embed, store, search) transfer directly to any real, persistent vector store covered in Module 24, so there’s no real benefit to adding that complexity before you need it.

What you should take away from this module

• The retrieval pipeline is: Documents → Loaders → Splitters → Embeddings → Vector Store → Retriever → Model, and each stage gets its own dedicated module next.
• A retriever is a genuine Runnable, using the same .invoke() interface as everything else since Module 8, composing directly into | chains.
• Real embeddings retrieve by semantic closeness, not shared words — the concrete advantage over Module 16’s placeholder search tool.

Where this goes next

The next module starts at the beginning of this pipeline: Document Loaders — bringing real files, from plain text to PDFs to web pages, into LangChain’s world as genuine Document objects.