TechByteByByte

Vector Databases

Why storing and searching millions of embedding vectors is a fundamentally different problem than a traditional database's exact-match lookup, and what a vector database actually stores.

#RAG#AI#Vector Databases#Level 3

Begin with the problem

Millions of vectors need storage, indexes, filters, updates, and backups. A vector database packages those operational jobs around similarity search.

query → vector/filters → index search → top candidates

What you will learn

  • Explain Vector Databases in simple language before using its technical details.
  • Follow the mechanism step by step through a small RAG example.
  • Connect this topic to the modules before and after it.
  • Decide when to use it, when not to use it, and what to measure in production.

Current real-system grounding: OpenAI’s vector store API and Google’s File Search guide are current examples of managed vector retrieval. Exact indexes and tuning controls vary by product.

The product example proves that the pattern is used in a real system. It does not mean every provider uses the same hidden algorithm, defaults, limits, or pricing.

1. The problem this module solves

Modules 10-11 established how to embed text and measure similarity between two vectors. This module addresses the next real question: where do you actually store potentially millions of these vectors, and how do you search them efficiently? This introduces vector databases — and sets up Module 13’s really important scaling problem.

2. The Problem — Comparing Against Everything Doesn’t Scale

Module 11 computed similarity between one query vector and a handful of chunk vectors — really trivial at that scale. Now imagine TechCorp’s knowledge base has grown to 10 million chunks.

Naive approach: for EVERY query, compute similarity against ALL 10
                million stored vectors, one by one, then sort

This is called BRUTE-FORCE search -- and it really works
CORRECTLY. It's just too SLOW to be usable at real scale, for every
single user question.

A vector database exists precisely to solve this scaling problem — Module 13 covers exactly how it solves it (approximate search); this module covers what a vector database actually is and stores.

3. Vector Database vs. Traditional Database

TRADITIONAL DATABASE:

WHERE employee_id = 123

An EXACT lookup -- either a row matches, or it doesn't. No concept
of "closeness."


VECTOR DATABASE:

"Find chunks semantically CLOSEST to this query vector."

A SIMILARITY search -- results are RANKED by how close they are, not
a binary match/no-match.

This is a really fundamental difference in what kind of question each database is built to answer. A traditional database answers “does this exact thing exist?” A vector database answers “what exists that’s most like this?”

4. What a Vector Database Actually Stores

Vector Database Entry:
├── The VECTOR itself (the embedding, from Module 10)
├── The ORIGINAL TEXT (the chunk's content)
└── METADATA (Module 9 -- document_id, section, access_control...)

A really important, often-missed point: vector databases don’t ONLY store vectors. They typically store the vector, the original text it came from, and its metadata all together — because when a search finds a matching vector, you need to actually retrieve the human-readable content and metadata that vector represents, not just a bare list of numbers.

5. A Real Developer Example

TechCorp stores its 8 travel-policy chunks (Module 9) in a vector
database.

Each ENTRY contains:
   - The chunk's embedding vector (e.g., [0.82, 0.48, 0.18, ...])
   - The chunk's actual TEXT ("London and Tokyo have a raised limit
     of $250 per night.")
   - Metadata: document_id="travel_policy_2026", section="4.2",
     access_control=["all_employees"]

When an employee's query vector is compared against these entries,
the database doesn't just return "vector #5 is closest" -- it
returns the FULL entry: text, document_id, section, and access
control, ready for citation (Module 23) and access checking (Module
27).

6. A Simple Agentic AI Connection

An agent’s “search knowledge base” tool is, mechanically, almost always a thin wrapper around a vector database query — the agent provides a search query, the tool embeds it (Module 10), searches the vector database, and returns the matched entries (text + metadata) back to the agent for it to reason over.

7. How Is This Used in AI?

🤖 How Is This Used in AI?

Vector databases are the standard, purpose-built infrastructure layer behind nearly every production RAG system’s retrieval step — chosen specifically for their ability to store embeddings alongside text and metadata, and to search across really large collections efficiently (Module 13’s indexing techniques make this efficiency possible).

8. Real-World Applications

  • Enterprise knowledge base search at really large document scale
  • Recommendation systems storing item embeddings for similarity lookup
  • Deduplication systems finding near-identical content

9. Common Mistakes

Incorrect idea: Assuming a vector database only stores raw vectors.

Why it is incorrect: As shown directly in Section 4, production use really requires storing text and metadata alongside each vector, not vectors in isolation.

Incorrect idea: Using brute-force comparison at real production scale.

Why it is incorrect: As shown directly in Section 2, this really doesn’t scale — Module 13 covers the actual solution.

Incorrect idea: Conflating “vector database” with “RAG” itself.

Why it is incorrect: As established back in Module 2, a vector database is one implementation tool for the retrieval step — not RAG as a whole.

10. Limitations

  • Vector databases add real infrastructure and operational complexity compared to simply storing text in a normal database — really worth it once similarity search at scale is actually needed
  • Not every knowledge source belongs in a vector database — Module 60 covers when structured data (like SQL) is really the better fit

11. Quick Reference — The Whole Idea in One Diagram

Embedding (Module 10)
   ↓
Vector Database
   ├── stores: vector + original text + metadata (Module 9)
   └── supports: similarity search (Module 11's metrics), not exact
                match

Traditional DB:      exact lookup
Vector DB:               ranked similarity search

12. Code — Building a Minimal Vector Store

🎯 Target of this example: implement Section 4-5’s real developer example directly — a minimal vector store class that really stores vector + text + metadata together, and returns complete entries (not bare vectors) on search, exactly as a real vector database would.

Example 1 — Simple

import numpy as np
from dataclasses import dataclass, field

@dataclass
class VectorEntry:
    """Exactly Section 4's required structure -- vector, text, AND
    metadata stored TOGETHER, not just a bare vector."""
    vector: np.ndarray
    text: str
    metadata: dict = field(default_factory=dict)

class MinimalVectorStore:
    """A deliberately minimal vector database -- demonstrates the
    CORE storage concept without any indexing optimization (Module
    13 addresses the scaling problem this doesn't yet solve)."""

    def __init__(self):
        self.entries: list[VectorEntry] = []

    def add(self, vector: np.ndarray, text: str, metadata: dict = None):
        self.entries.append(VectorEntry(vector=vector, text=text, metadata=metadata or {}))

store = MinimalVectorStore()
store.add(
    vector=np.array([0.82, 0.48, 0.18]),
    text="London and Tokyo have a raised limit of $250 per night.",
    metadata={"document_id": "travel_policy_2026", "section": "4.2"},
)

print(f"Stored {len(store.entries)} entries")
print(f"Entry text: {store.entries[0].text}")
print(f"Entry metadata: {store.entries[0].metadata}")

Expected Output:

Stored 1 entries
Entry text: London and Tokyo have a raised limit of $250 per night.
Entry metadata: {'document_id': 'travel_policy_2026', 'section':
'4.2'}

What we conclude from this example: even this minimal class stores the vector, text, and metadata together as one unit — exactly Section 4’s requirement, giving downstream code everything it needs from a single retrieved entry.

Example 2 — Intermediate

import numpy as np
from dataclasses import dataclass, field

@dataclass
class VectorEntry:
    vector: np.ndarray
    text: str
    metadata: dict = field(default_factory=dict)

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

class MinimalVectorStore:
    def __init__(self):
        self.entries: list[VectorEntry] = []

    def add(self, vector: np.ndarray, text: str, metadata: dict = None):
        self.entries.append(VectorEntry(vector=vector, text=text, metadata=metadata or {}))

    def search(self, query_vector: np.ndarray, top_n: int = 3) -> list:
        """Section 2's BRUTE-FORCE approach -- compares against
        EVERY stored entry. Correct, but really doesn't scale
        (Module 13 addresses this directly)."""
        scored = [(entry, cosine_similarity(query_vector, entry.vector)) for entry in self.entries]
        scored.sort(key=lambda x: x[1], reverse=True)
        return scored[:top_n]

store = MinimalVectorStore()
store.add(np.array([0.82, 0.48, 0.18]),
          "London and Tokyo have a raised limit of $250 per night.",
          {"document_id": "travel_policy_2026", "section": "4.2"})
store.add(np.array([0.1, 0.9, 0.05]),
          "The office parking garage closes at 10pm.",
          {"document_id": "facilities_faq", "section": "1.0"})
store.add(np.array([0.79, 0.51, 0.21]),
          "Standard hotel limit is $200 per night.",
          {"document_id": "travel_policy_2026", "section": "4.1"})

query_vector = np.array([0.80, 0.50, 0.20])
results = store.search(query_vector, top_n=2)
for entry, score in results:
    print(f"[{score:.3f}] ({entry.metadata['document_id']} / {entry.metadata['section']}) {entry.text}")

Expected Output:

[1.000] (travel_policy_2026 / 4.1) Standard hotel limit is $200 per
night.
[0.999] (travel_policy_2026 / 4.2) London and Tokyo have a raised
limit of $250 per night.

What we conclude from this example: search() returns full entries — text AND metadata — not bare vectors or scores alone. This is exactly what makes vector database results immediately usable for citation and access control downstream, verified directly in the printed output showing document_id and section alongside each result.

Example 3 — Production Grade

import numpy as np
import time
from dataclasses import dataclass, field

@dataclass
class VectorEntry:
    vector: np.ndarray
    text: str
    metadata: dict = field(default_factory=dict)

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

class MinimalVectorStore:
    def __init__(self):
        self.entries: list[VectorEntry] = []

    def add(self, vector: np.ndarray, text: str, metadata: dict = None):
        self.entries.append(VectorEntry(vector=vector, text=text, metadata=metadata or {}))

    def search(self, query_vector: np.ndarray, top_n: int = 3) -> list:
        scored = [(entry, cosine_similarity(query_vector, entry.vector)) for entry in self.entries]
        scored.sort(key=lambda x: x[1], reverse=True)
        return scored[:top_n]

def benchmark_brute_force_scaling(entry_counts: list) -> dict:
    """Directly measures Section 2's scaling claim -- how search time
    grows as the number of stored vectors grows, using brute-force
    search. This is the CONCRETE motivation for Module 13's indexing
    techniques."""
    results = {}
    rng = np.random.default_rng(42)

    for count in entry_counts:
        store = MinimalVectorStore()
        for i in range(count):
            store.add(rng.normal(0, 1, size=64), f"chunk_{i}", {"id": i})

        query_vector = rng.normal(0, 1, size=64)
        start = time.time()
        store.search(query_vector, top_n=5)
        elapsed = time.time() - start
        results[count] = round(elapsed, 5)

    return results

timings = benchmark_brute_force_scaling([1_000, 10_000, 50_000])
print("Brute-force search time vs. number of stored vectors:")
for count, elapsed in timings.items():
    print(f"  {count:,} vectors: {elapsed}s")

Expected Output:

Brute-force search time vs. number of stored vectors:
  1,000 vectors: 0.00597s
  10,000 vectors: 0.06258s
  50,000 vectors: 0.32909s

Note: exact timings vary by machine and run, but the GROWTH TREND
is the key, reproducible signal -- search time scales up roughly in
proportion to the number of stored vectors.

What we conclude from this example: search time grows roughly linearly with the number of stored vectors — at 50,000 vectors, it’s already meaningfully slower than at 1,000, and this trend would continue to worsen at millions or billions of vectors. This is the concrete, measurable version of Section 2’s scaling problem, and it’s precisely the motivation for Module 13’s approximate nearest neighbor techniques — no production vector database relies on pure brute-force search at real scale.

13. Interview Questions

Q: What’s the fundamental difference between what a traditional database and a vector database are each built to answer?

Ans: A traditional database answers exact-match lookup questions — does a row matching these specific criteria exist? A vector database answers a really different kind of question — what stored content is most similar to this query, ranked by closeness in vector space, rather than a binary match or no-match. This reflects a fundamentally different underlying problem: exact retrieval versus similarity-based ranking.

Q: Why does a production vector database need to store more than just the raw embedding vectors?

Ans: When a similarity search finds a matching vector, the system needs to actually return something useful to the application — the original text that vector represents, and metadata like document source, section, and access permissions. Storing only bare vectors with no associated content or metadata would make search results mathematically correct but practically useless, since there’d be no way to retrieve what that vector actually meant or where it came from.

Q: Why does brute-force similarity search become a real problem at production scale, even though it’s mathematically correct?

Ans: Brute-force search compares a query vector against every single stored vector, one at a time, to find the closest matches. This approach is correct but scales roughly linearly with the number of stored vectors — as a knowledge base grows from thousands to millions of chunks, search time grows correspondingly, eventually becoming too slow to serve real-time user queries. This is a real, measurable scaling problem that specialized indexing techniques exist specifically to solve.

Q: If you were debugging a RAG system where retrieved chunks were missing important context needed to answer a citation question, what would you check about how the vector database was storing entries?

Ans: I’d check whether the vector database entries were storing complete metadata alongside each vector and its text — specifically, whether document_id, section, and other citation-relevant fields were actually present and populated on each stored entry. If metadata wasn’t captured correctly during ingestion (Module 5) or wasn’t properly carried through chunking (Module 9) before being stored, the vector database would have nothing to return for citation purposes even if the actual similarity search itself was working correctly.

14. What You Should Remember

  • Vector databases answer similarity-ranked search, not exact-match lookup — a really different kind of question than a traditional database.
  • Vector database entries store vector + text + metadata together — verified directly by a minimal implementation returning complete, citable entries on search, not bare vectors.
  • Brute-force search doesn’t scale — verified directly by measuring search time growth as stored vector count increases, motivating Module 13’s indexing techniques.

15. Quick Practice

Explain, in your own words, why storing embeddings in a vector database without their original text would make the database practically useless for a RAG system, even if the similarity math itself worked perfectly.

16. Next Step

Next: Module 13 — Vector Indexing & ANN — the actual solution to this module’s scaling problem: approximate nearest neighbor search, and why trading a small amount of accuracy for massive speed gains is usually the right choice.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed