Retrievers: Beyond as_retriever()

You’ve called .as_retriever() already, in Module 21’s preview. This module goes deeper — real configuration options, and a genuinely important architectural decision this course has been quietly building toward since Module 16’s placeholder search tool.

Example 1: controlling how many results come back

from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings
from langchain_core.vectorstores import InMemoryVectorStore

docs = [
    Document(page_content="Our return policy allows returns within 30 days.", metadata={"category": "returns"}),
    Document(page_content="Refunds are processed within 5-7 business days.", metadata={"category": "returns"}),
    Document(page_content="Shipping takes 3-5 business days.", metadata={"category": "shipping"}),
]

vector_store = InMemoryVectorStore(OpenAIEmbeddings(model="text-embedding-3-small"))
vector_store.add_documents(docs)

retriever = vector_store.as_retriever(search_kwargs={"k": 2})
results = retriever.invoke("Tell me about returns and refunds.")

for doc in results:
    print(doc.page_content)

k controls exactly how many chunks come back. Recall Module 23’s real trade-off around chunk size — k carries an analogous one: too few results risks missing genuinely relevant content; too many dilutes the model’s attention with marginally relevant material, exactly the concern the RAG evaluation concepts from later in your broader curriculum measure directly.

Example 2: filtering by metadata

Recall Module 22’s metadata — this is exactly where it earns its keep.

retriever = vector_store.as_retriever(
    search_kwargs={"k": 2, "filter": {"category": "returns"}}
)

results = retriever.invoke("How long do things take?")
for doc in results:
    print(doc.page_content, "-", doc.metadata)

Even though the query is genuinely ambiguous — it could match shipping content just as easily — the metadata filter constrains the search to only the "returns" category from the start. This is a real, deliberate way to narrow retrieval using information your application already knows, rather than relying purely on semantic similarity to guess correctly.

Example 3: a different search strategy — maximum marginal relevance

retriever = vector_store.as_retriever(
    search_type="mmr",
    search_kwargs={"k": 2, "fetch_k": 3},
)

results = retriever.invoke("Tell me about your policies.")
for doc in results:
    print(doc.page_content)

Plain similarity search can return several results that are all genuinely relevant but also genuinely redundant — saying nearly the same thing. "mmr" (maximum marginal relevance) actively favors diversity among its results, alongside relevance — genuinely useful when you want a broad, varied picture rather than several near-duplicate chunks all making the same point.

Example 4: a retriever as a fixed pipeline step

This is the shape from every RAG example you’ve implicitly been building toward.

from langchain.chat_models import init_chat_model
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough

retriever = vector_store.as_retriever(search_kwargs={"k": 2})
model = init_chat_model("openai:gpt-4o-mini")
parser = StrOutputParser()

prompt = ChatPromptTemplate.from_template(
    "Answer the question using only this context:\n{context}\n\nQuestion: {question}"
)

def format_docs(docs):
    return "\n\n".join(d.page_content for d in docs)

chain = (
    RunnablePassthrough.assign(context=lambda x: format_docs(retriever.invoke(x["question"])))
    | prompt
    | model
    | parser
)

result = chain.invoke({"question": "How long do refunds take?"})
print(result)

Notice RunnablePassthrough.assign(...) from Module 9, doing exactly the job it was built for — carrying the original question forward while adding context, computed by actually running the retriever. In this shape, retrieval always runs, on every single call, whether or not the question genuinely needs it.

Example 5: a retriever as an agent tool

Recall Module 16’s Agent 6, and its explicit promise: swap the crude placeholder for a real retriever, keep the rest of the pattern. Here’s that promise, kept.

from langchain.chat_models import init_chat_model
from langchain.tools import tool
from langchain.agents import create_agent

@tool
def search_policies(query: str) -> str:
    """Search company policy documents for relevant information."""
    results = retriever.invoke(query)
    return "\n\n".join(d.page_content for d in results) if results else "No relevant policy found."

agent = create_agent(model=init_chat_model("openai:gpt-4o-mini"), tools=[search_policies])

result = agent.invoke({"messages": [{"role": "user", "content": "What's 2+2? Also, what's your return policy?"}]})
print(result["messages"][-1].content)

Now retrieval only runs when the agent actually decides it’s needed. Ask it “what’s 2+2,” and it never calls search_policies at all — it answers directly. This is the real, meaningful architectural choice worth naming explicitly:

flowchart LR
    A["Fixed RAG pipeline:\nretrieval ALWAYS runs"] --- B["Agentic retrieval:\nmodel decides IF retrieval is needed"]

A fixed pipeline is simpler, more predictable, and genuinely appropriate when every request truly needs retrieval. Agentic retrieval costs one extra decision but avoids wasted, irrelevant retrieval on questions that don’t need it at all — exactly the same “does this really need another LLM call” thinking that’s been a quiet, recurring theme since Module 12.

Common mistakes worth avoiding

Setting k too low and silently missing genuinely relevant content. Recall this module’s own real trade-off discussion — k=1 might work fine on simple test questions and then quietly fail the moment a real question needs information spread across two different chunks. Test with k values genuinely representative of your real content’s complexity, not just the simplest case.

Reaching for search_type="mmr" by default, without a genuine need for diversity. Recall Example 3 — MMR is a real, deliberate trade-off, sacrificing some pure relevance for variety. For a query with one clear, correct answer, plain similarity search is often both simpler and more directly accurate.

Building a fixed RAG pipeline (Example 4) when the application genuinely needs agentic retrieval (Example 5), or vice versa. Recall this module’s own closing distinction — defaulting to whichever pattern you learned first, rather than deliberately choosing based on whether every real request genuinely needs retrieval, is an easy, avoidable mistake with real cost and complexity consequences either way.

What you should take away from this module

• k and metadata filter are real, practical controls over what a retriever actually returns — worth tuning deliberately, not left at defaults.
• search_type="mmr" favors diverse results over near-duplicate ones, genuinely useful when a query could reasonably match several similar chunks.
• A retriever as a fixed pipeline step always runs, using RunnablePassthrough.assign() — simple, predictable, right when every request needs retrieval.
• A retriever as an agent tool only runs when the model decides it’s genuinely needed — this is the real fix for Module 16’s placeholder, promised and now delivered.

Where this goes next

The next module builds a complete, real RAG system from the ground up, version by version — manual retrieval, the retriever abstraction, structured responses, citations, and conversation-aware RAG, bringing every piece from this entire retrieval sequence together into one working application.