TechByteByByte

Agentic RAG & Graph RAG

Taking Self-RAG and Corrective RAG's reasoning further into really autonomous, multi-step retrieval, and introducing graph-based retrieval for relationship-heavy questions vector similarity cannot answer.

#RAG#AI#Agentic RAG#Graph RAG#Level 7

Begin with the problem

Some questions require several searches, decisions, or relationship hops. Agentic RAG and Graph RAG add planning or graph structure, but also add failure modes and cost.

question → choose specialized retrieval path → collect multimodal/structured evidence → answer

What you will learn

  • Explain Agentic RAG & Graph RAG in simple language before using its technical details.
  • Follow the mechanism step by step through a small RAG example.
  • Connect this topic to the modules before and after it.
  • Decide when to use it, when not to use it, and what to measure in production.

Current real-system grounding: Google’s current File Search documentation includes file-based grounding and multimodal retrieval capabilities, with model, file-type, and tool-combination limitations.

The product example proves that the pattern is used in a real system. It does not mean every provider uses the same hidden algorithm, defaults, limits, or pricing.

1. The problem this module solves

Module 28 introduced systems that reason about their own retrieval quality. This module covers two further extensions: giving an agent real autonomy over the entire retrieval process (Agentic RAG), and addressing a really different kind of question — one about relationships between entities — that vector similarity alone cannot answer (Graph RAG).

2. Agentic RAG — From Fixed Pipeline to Real Reasoning

Traditional RAG (even Module 28's Corrective RAG):

Question -> Retrieve -> [maybe evaluate, maybe retry] -> Answer

AGENTIC RAG:

Goal
   ↓
REASON about what's really needed
   ↓
CHOOSE a retrieval strategy (vector search? BM25? SQL query, Module
                             30? multiple searches?)
   ↓
Search
   ↓
EVALUATE the results
   ↓
Search AGAIN if really necessary (possibly with a DIFFERENT
                                     strategy or query)
   ↓
Generate

This is really your Generative AI course’s agent loop (Module 29 of that course), applied specifically to retrieval. The difference from Module 28’s Corrective RAG isn’t the CONCEPT (both involve evaluating and retrying) — it’s the DEGREE of autonomy: an agent can really choose BETWEEN different retrieval TOOLS and strategies, not just retry the SAME method with a different query.

3. When Agentic Retrieval Is Really Useful

Simple question: "What's the London hotel limit?"
   -> A SINGLE, direct vector search really suffices -- agentic
      overhead adds NOTHING here

Complex question: "Compare our Q3 revenue growth to our
                   competitors' publicly reported figures, and
                   explain the difference"
   -> REALLY needs: (1) an internal SQL query for Q3 revenue
      (Module 30), (2) a WEB SEARCH for competitor figures (outside
      the internal knowledge base entirely), (3) REASONING to
      compare and explain the two

Agentic RAG’s real value emerges specifically for questions where the right retrieval STRATEGY isn’t known in advance, or really requires combining MULTIPLE different sources or methods.

4. Graph RAG — A Really Different Kind of Question

Recall Module 11: vector similarity is excellent at finding semantically SIMILAR content. But some questions aren’t fundamentally about similarity at all — they’re about RELATIONSHIPS.

"Which employees worked on projects that used technology X and
were managed by department Y?"

This is REALLY not a "find similar text" question -- it's a
MULTI-HOP RELATIONSHIP question: Employee -> works_on -> Project ->
uses -> Technology, AND Project -> managed_by -> Department
Entities (Alice, Project Falcon, Kubernetes, Engineering)
   +
Relationships (works_on, uses_technology, managed_by)
   ↓
KNOWLEDGE GRAPH
   ↓
Multi-hop TRAVERSAL (following relationships step by step) --
REALLY different from vector similarity search
   ↓
Answer

5. Why Vector Search Really Struggles With Relationship

Questions

A vector embedding of "Alice works on Project Falcon which uses
Kubernetes" captures the GENERAL semantic content of that sentence.

But it does NOT really encode a QUERYABLE relationship structure
-- you can't reliably ask "find all X such that X relates to Y via
relationship Z" using SIMILARITY alone. Embeddings capture MEANING;
graphs capture STRUCTURE.

Graph RAG doesn’t replace vector search — it’s a really complementary approach for the specific class of questions that require traversing explicit relationships, not just finding semantically similar content.

6. A Real Developer Example

TechCorp builds an internal "who knows what" assistant:

Vector-search-based question: "Find documentation about Kubernetes
                               deployment best practices"
   -> Standard semantic retrieval (Modules 10-18) works really
      well here

Graph-based question: "Which employees have worked on projects using
                       Kubernetes AND report to the Platform
                       Engineering department?"
   -> Requires traversing: Employee -> works_on -> Project ->
      uses_technology -> Kubernetes, AND Employee -> reports_to ->
      Platform Engineering -- a REALLY different retrieval
      mechanism than similarity search

A SOPHISTICATED system might use BOTH: Graph RAG to identify the
RIGHT set of employees, then VECTOR search across THEIR specific
project documentation for more detailed context.

7. A Simple Agentic AI Connection

An agent equipped with BOTH a vector search tool and a graph query tool can really choose the right one based on the nature of a given sub-question — recognizing when a question is fundamentally about “finding similar content” versus “traversing known relationships,” directly connecting this module’s two techniques to Module 29 of your Generative AI course’s tool-selection reasoning.

8. How Is This Used in AI?

🤖 How Is This Used in AI?

Agentic RAG powers increasingly sophisticated research and analysis assistants that need to combine multiple information sources and strategies dynamically. Graph RAG is used specifically in domains with really rich, explicit relationship structures — organizational charts, product dependency graphs, scientific citation networks — where relationship-aware queries are a core, recurring need.

9. Real-World Applications

  • Agentic RAG: research assistants, complex multi-source business intelligence questions
  • Graph RAG: organizational knowledge (“who knows what”), scientific literature citation analysis, product/system dependency mapping

10. Common Mistakes

Incorrect idea: Using agentic RAG for really simple questions.

Why it is incorrect: As shown directly in Section 3, this adds real overhead without real benefit for straightforward, single-source questions.

Incorrect idea: Trying to answer relationship questions with pure vector search.

Why it is incorrect: As shown directly in Section 5, embeddings capture meaning, not queryable relationship structure — really the wrong tool for this class of question.

Incorrect idea: Assuming Graph RAG replaces vector search entirely.

Why it is incorrect: As shown directly in Section 5-6, these are complementary techniques for really different question types, not competitors.

11. Limitations

  • Agentic RAG really adds latency and cost (Module 25, 27 of the Generative AI course) from multiple reasoning and retrieval steps
  • Building and maintaining a real knowledge graph requires real, ongoing effort to keep entities and relationships current and accurate — a real, additional data-maintenance burden beyond standard document ingestion

12. Quick Reference — The Whole Idea in One Diagram

AGENTIC RAG:      Goal -> reason -> CHOOSE strategy -> search ->
                 evaluate -> search AGAIN if needed (possibly
                 DIFFERENT strategy) -> generate

GRAPH RAG:            Entities + Relationships -> Knowledge Graph ->
                    MULTI-HOP traversal -> answers RELATIONSHIP
                    questions vector similarity cannot

13. Code — Implementing Multi-Hop Graph Traversal

🎯 Target of this example: implement Section 4 and 6’s real developer example directly — building a small knowledge graph and answering a real multi-hop relationship question (“which employees work on projects using Kubernetes”) that vector similarity search alone could not reliably answer.

Example 1 — Simple

class SimpleKnowledgeGraph:
    """A minimal, illustrative knowledge graph -- nodes are entities,
    edges are labeled relationships, exactly Section 4's structure."""

    def __init__(self):
        self.edges = {}

    def add_relationship(self, source: str, relationship: str, target: str):
        self.edges.setdefault(source, []).append((relationship, target))

    def query(self, entity: str) -> list:
        return self.edges.get(entity, [])

graph = SimpleKnowledgeGraph()
graph.add_relationship("Alice", "works_on", "Project Falcon")
graph.add_relationship("Project Falcon", "uses_technology", "Kubernetes")

print("Alice's relationships:", graph.query("Alice"))
print("Project Falcon's relationships:", graph.query("Project Falcon"))

Expected Output:

Alice's relationships: [('works_on', 'Project Falcon')]
Project Falcon's relationships: [('uses_technology', 'Kubernetes')]

What we conclude from this example: the graph correctly stores and returns each entity’s direct relationships — the foundational structure Section 4 described, ready for real multi-hop traversal in the next example.

Example 2 — Intermediate

class SimpleKnowledgeGraph:
    def __init__(self):
        self.edges = {}

    def add_relationship(self, source: str, relationship: str, target: str):
        self.edges.setdefault(source, []).append((relationship, target))

    def find_entities_with_relationship(self, relationship: str, target: str) -> list:
        """Reverse lookup: which entities have THIS relationship TO
        this target? (e.g., 'who works_on Project Falcon?') --
        exactly the operation multi-hop traversal needs."""
        return [source for source, rels in self.edges.items()
                for rel, tgt in rels if rel == relationship and tgt == target]

graph = SimpleKnowledgeGraph()
graph.add_relationship("Alice", "works_on", "Project Falcon")
graph.add_relationship("Bob", "works_on", "Project Falcon")
graph.add_relationship("Project Falcon", "uses_technology", "Kubernetes")
graph.add_relationship("Charlie", "works_on", "Project Eagle")
graph.add_relationship("Project Eagle", "uses_technology", "Docker")

# Section 6's exact question: "Which employees work on projects that
# use Kubernetes?" -- a real TWO-HOP traversal.
projects_using_kubernetes = graph.find_entities_with_relationship("uses_technology", "Kubernetes")
print(f"Projects using Kubernetes: {projects_using_kubernetes}")

employees = []
for project in projects_using_kubernetes:
    employees.extend(graph.find_entities_with_relationship("works_on", project))
print(f"Employees working on those projects: {employees}")

Expected Output:

Projects using Kubernetes: ['Project Falcon']
Employees working on those projects: ['Alice', 'Bob']

What we conclude from this example: this really answers Section 6’s relationship question correctly — Alice and Bob, both working on Project Falcon (which uses Kubernetes), are identified, while Charlie (working on the Docker-based Project Eagle) is correctly excluded. This is precisely the kind of multi-hop relationship question Section 5 explained vector similarity search cannot reliably answer.

Example 3 — Production Grade

from dataclasses import dataclass

@dataclass
class GraphQueryResult:
    query_description: str
    matched_entities: list
    hops_traversed: int

class KnowledgeGraph:
    """A more complete, production-style knowledge graph supporting
    GENERIC multi-hop path traversal -- Section 4's structure,
    extended to handle ARBITRARY relationship chains, not just one
    hardcoded two-hop case."""

    def __init__(self):
        self.edges = {}

    def add_relationship(self, source: str, relationship: str, target: str):
        self.edges.setdefault(source, []).append((relationship, target))

    def _reverse_lookup(self, relationship: str, target: str) -> list:
        return [source for source, rels in self.edges.items()
                for rel, tgt in rels if rel == relationship and tgt == target]

    def multi_hop_query(self, start_relationship: str, start_target: str,
                         additional_hops: list = None) -> GraphQueryResult:
        """Traverses a chain of relationships STARTING from a known
        target, working BACKWARD through each hop -- a really
        reusable pattern for answering relationship questions of
        varying complexity."""
        current_entities = self._reverse_lookup(start_relationship, start_target)
        hops = 1

        for relationship in (additional_hops or []):
            next_entities = []
            for entity in current_entities:
                next_entities.extend(self._reverse_lookup(relationship, entity))
            current_entities = next_entities
            hops += 1

        return GraphQueryResult(
            query_description=f"{start_relationship} -> {start_target}" +
                              (f" -> {' -> '.join(additional_hops)}" if additional_hops else ""),
            matched_entities=current_entities, hops_traversed=hops,
        )

graph = KnowledgeGraph()
graph.add_relationship("Alice", "works_on", "Project Falcon")
graph.add_relationship("Bob", "works_on", "Project Falcon")
graph.add_relationship("Project Falcon", "uses_technology", "Kubernetes")
graph.add_relationship("Project Falcon", "managed_by", "Platform Engineering")
graph.add_relationship("Charlie", "works_on", "Project Eagle")
graph.add_relationship("Project Eagle", "uses_technology", "Docker")

# "Which projects use Kubernetes, and who works on them?" -- a
# real 2-hop query, implemented generically.
result = graph.multi_hop_query("uses_technology", "Kubernetes", additional_hops=["works_on"])

print(f"Query: {result.query_description}")
print(f"Hops traversed: {result.hops_traversed}")
print(f"Matched entities: {result.matched_entities}")

Expected Output:

Query: uses_technology -> Kubernetes -> works_on
Hops traversed: 2
Matched entities: ['Alice', 'Bob']

What we conclude from this example: the generic multi_hop_query method correctly answers the same relationship question as Example 2, but now as a reusable, parameterized function that could handle really different relationship chains without hardcoding the specific hops — exactly the kind of production-ready graph traversal capability a real Graph RAG system needs, directly extending Section 6’s real developer example into reusable infrastructure.

14. Interview Questions

Q: What really distinguishes Agentic RAG from Corrective RAG, given that both involve evaluating and retrying retrieval?

Ans: Corrective RAG retries with the same underlying retrieval method but a possibly different query if the initial attempt was insufficient. Agentic RAG involves real autonomy over choosing between fundamentally different retrieval strategies or tools — vector search, keyword search, a SQL query, or even an external web search — based on reasoning about what a specific question actually requires. The difference is the degree of autonomy: Agentic RAG can choose between different approaches entirely, not just retry the same approach differently.

Q: Explain why a question like “which employees worked on projects using technology X and managed by department Y” is really difficult for vector similarity search to answer well.

Ans: This is fundamentally a multi-hop relationship question — it requires traversing explicit connections between entities (employee to project, project to technology, project to department), not finding content that’s semantically similar to the question’s wording. A vector embedding captures the general meaning of text, but doesn’t encode a queryable relationship structure that lets you reliably ask “find all X such that X relates to Y via relationship Z.” Graph-based retrieval, which explicitly models entities and their relationships, is really better suited to this class of question.

Q: How does Graph RAG’s multi-hop traversal actually work, mechanically?

Ans: A knowledge graph stores entities and labeled relationships between them. To answer a multi-hop question, the system starts from a known entity or value, follows a specific relationship to find connected entities, and then follows additional relationships from those results to continue the chain — for example, finding all projects that use a specific technology, then finding all employees who work on those specific projects. Each “hop” traverses one relationship step, and the final set of entities after all hops represents the answer.

Q: Does Graph RAG replace vector-based semantic search? Explain your reasoning.

Ans: No — they’re complementary techniques suited to really different question types. Vector search excels at finding content that’s semantically similar to a query, even when exact relationships aren’t explicitly known or queried. Graph RAG excels at answering questions that depend on explicit, structured relationships between entities. A sophisticated system often uses both together — for example, using Graph RAG to identify a relevant set of entities through relationship traversal, then using vector search to find detailed, semantically relevant content specifically about those identified entities.

15. What You Should Remember

  • Agentic RAG gives real autonomy to choose between different retrieval strategies based on reasoning about what a question actually needs — a further extension of Module 28’s Corrective RAG.
  • Graph RAG answers relationship-based questions that vector similarity search cannot reliably handle — verified directly through a working multi-hop traversal correctly answering “which employees work on projects using Kubernetes.”
  • These techniques are complementary, not competing — verified directly by recognizing which question type each is really suited to.

16. Quick Practice

Design a knowledge graph schema (entities and relationship types) for a scenario in your own domain of interest, and write out one real multi-hop question that this graph structure could answer but a pure vector similarity search likely could not.

17. Next Step

Next: Module 30 — RAG with Structured Data & Tables — recognizing when the right answer to “where should this data live” is a SQL database, not a vector store at all.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed