Section 6 — Vector Store

Begin with the problem

Finding nearby meanings quickly

After text becomes vectors, an application needs somewhere to store them and find the closest matches. VectorStore gives Spring applications a shared way to perform that search.

query → embedding → similarity search → matching documents

What you will learn

• Store documents and embeddings.
• Run similarity search with metadata filters.
• Explain why indexes trade exactness for speed.
• Choose a vector store using scale and operational needs.

Current official reference: Spring AI documentation for this topic. The examples below primarily preserve the stated 1.1.x course target. Where Spring AI 2.0 differs, the text must treat that behavior as version-specific rather than universal.

(Continues from Section 5. Target: Spring AI 1.1.x / Spring Boot 3.5.x.)

6.1 Why VectorStore Is Its Own Abstraction, Not Just “a Repository”

Beginner primer: a vector database (or vector store) is a database purpose-built (or extended) to store embedding vectors and answer “find me the N vectors closest to this query vector” efficiently, even across millions of entries — a plain SQL WHERE clause doesn’t scale to that kind of nearest-neighbor search. If you haven’t used one before, think of it as analogous to how a search engine’s inverted index makes keyword search fast — a vector index makes similarity search fast.

What’s architecturally interesting is that Spring AI’s VectorStore interface unifies wildly different backends — a Postgres extension (pgvector), a dedicated vector-native database (Qdrant, Milvus, Pinecone), a repurposed general-purpose store (Redis, Elasticsearch, MongoDB Atlas) — behind one contract that also owns the embedding step internally, not just the storage/query step.

Real-world analogy — Hospital Blood Bank: A blood bank doesn’t just store blood; it types it (embeds it), cross-references compatibility on retrieval (similarity search), and applies eligibility filters (metadata filtering — donor age, screening status). VectorStore.add(List<Document>) is the intake-and-typing process; similaritySearch(SearchRequest) is a cross-match request with filters attached — you don’t hand the blood bank raw untyped blood and ask it to also do the typing separately from storage; it’s one integrated operation, same as VectorStore.add() calling EmbeddingModel internally as part of storing a Document.

Analogy: The Hospital Blood Bank Matcher Imagine running a specialized blood bank inside an emergency hospital:

• Ingestion (The Intake Typing): When blood donations arrive, you don’t just stack unlabelled vials in a fridge. The intake desk types each vial (calls the EmbeddingModel to generate a 1536-dimensional coordinate vector), assigns it compatibility labels, and files it in a catalog with donor properties (metadata: donor age, region, test date). This is VectorStore.add().
• Retrieval (The Compatibility Search): An emergency call comes in: “Need 3 compatibility-matching vials for a patient, but only from donors screened in region ‘IN’.”
• You don’t scan all 100,000 vials manually. The blood bank system instantly filters the list to region "IN" first (metadata filtering), then computes the closest physical matches (similarity search) to the patient’s blood type vector.

📊 Visual Flowchart: pgvector Database Similarity Search Trace

Here is how a generic SearchRequest compiles into a cosine-distance SQL query with JSONB filtering:

graph TD
    Request["SearchRequest:<br>query='refund policy', topK=5, region='IN'"] --> Transform["1. SearchRequest Compiler<br>(Translate filter expression to SQL)"]
    Transform --> SQL["2. Generated SQL Query:<br>SELECT id, content, 1 - (embedding <=> :queryVec) AS similarity<br>FROM vector_store<br>WHERE metadata->>'region' = 'IN'<br>ORDER BY embedding <=> :queryVec LIMIT 5"]

    SQL --> Exec["3. JDBC Execute against Postgres pgvector"]
    Exec --> RowFilter{"4. Similarity >= Threshold?"}

    RowFilter -->|Yes| Doc["Wrap as Document list"]
    RowFilter -->|No| Exclude["Discard document chunk"]

---

6.2 The Interface

public interface VectorStore extends DocumentWriter {

    void add(List<Document> documents);

    void delete(List<String> idList);
    void delete(Filter.Expression filterExpression);

    List<Document> similaritySearch(String query);
    List<Document> similaritySearch(SearchRequest request);
}

SearchRequest is the real workhorse object:

SearchRequest request = SearchRequest.builder()
        .query("What is our refund policy for defective electronics?")
        .topK(5)
        .similarityThreshold(0.75)
        .filterExpression("category == 'returns' && region == 'IN'")
        .build();

List<Document> results = vectorStore.similaritySearch(request);

Beginner note: topK means “return the K closest matches” (here, the 5 most similar documents). similarityThreshold is a minimum cosine-similarity cutoff (see Section 5) — results below this score are excluded even if they’d otherwise be in the top 5, which matters a great deal (§6.7, §6.8).

6.2.1 What Happens Inside add(List<Document>)

vectorStore.add(documents)
        │
        ▼
1. For each Document lacking a pre-computed embedding:
   EmbeddingModel.embed(document.getContent()) called —
   usually BATCHED internally by well-implemented VectorStore backends
   (check specific implementation; not all batch equally well —
   this is a real per-implementation performance difference worth
   verifying against the version you're running)
        │
        ▼
2. Document.getMetadata() (arbitrary Map<String,Object>) preserved
   alongside the vector — this is what powers metadata filtering later
        │
        ▼
3. Backend-specific write: pgvector → INSERT with vector column,
   Pinecone → upsert() API call, Redis → HSET with vector field,
   each translated from the generic Document+embedding pair

---

6.3 Supported Vector Stores — Comparison Matrix

Store	Module	Underlying tech	Metadata filtering	Hybrid search	Best fit
PGVector	spring-ai-pgvector-store	Postgres extension	SQL WHERE via Filter.Expression translated to SQL	Via combining pgvector with Postgres full-text search (tsvector) manually	Teams already running Postgres who want one database for relational + vector data, avoiding new infra
Redis	spring-ai-redis-store	RediSearch module	Yes, via RediSearch query syntax	Yes, RediSearch supports hybrid natively	Low-latency use cases, teams already running Redis
Pinecone	spring-ai-pinecone-store	Managed vector-native SaaS	Yes, metadata filter DSL	Limited (sparse-dense hybrid on newer index types)	Teams wanting zero infra ops, willing to pay for managed service
Qdrant	spring-ai-qdrant-store	Vector-native, self-hosted or cloud	Rich filtering (nested, geo, range)	Yes	Teams wanting powerful filtering without a managed-SaaS bill
Milvus	spring-ai-milvus-store	Vector-native, built for massive scale	Yes	Yes	Very large-scale (billions of vectors) deployments
Weaviate	spring-ai-weaviate-store	Vector-native, GraphQL API	Yes	Native hybrid (BM25 + vector)	Teams wanting hybrid search as a first-class, well-documented feature
Elasticsearch	spring-ai-elasticsearch-store	Search engine with vector extensions (kNN)	Yes, full Elasticsearch query DSL	Excellent — Elasticsearch’s core strength is text search, vector is additive	Teams already running Elasticsearch for logs/search who want to consolidate
MongoDB Atlas	spring-ai-mongodb-atlas-store	Atlas Vector Search	Yes, MQL-based	Yes (Atlas Search + Vector Search combined)	Teams already on MongoDB Atlas

The decision that matters most in practice: if you already operate one of these systems in production (Postgres, Redis, Elasticsearch, MongoDB), the “best” vector store is almost always the one you already run, not the one with the best benchmark numbers — avoiding a new piece of infrastructure to operate, monitor, back up, and secure is worth more than marginal recall/latency differences for the vast majority of production RAG systems below billion-vector scale.

---

6.4 Internal Architecture — PGVector Deep Dive (Representative Example)

@Bean
public VectorStore vectorStore(JdbcTemplate jdbcTemplate, EmbeddingModel embeddingModel) {
    return PgVectorStore.builder(jdbcTemplate, embeddingModel)
            .dimensions(1536)
            .distanceType(PgDistanceType.COSINE_DISTANCE)
            .indexType(PgIndexType.HNSW)
            .initializeSchema(true)   // creates table + extension in dev; disable in production, see §6.8/6.11
            .build();
}

Table: vector_store
┌───────────┬──────────────┬───────────────┬────────────────┐
│ id (uuid) │ content (text) │ metadata (jsonb) │ embedding (vector(1536)) │
└───────────┴──────────────┴───────────────┴────────────────┘

similaritySearch(SearchRequest) internally builds:

SELECT id, content, metadata,
       1 - (embedding <=> :queryEmbedding) AS similarity
FROM vector_store
WHERE metadata @> :jsonbFilterFragment  -- from Filter.Expression translation
  AND (1 - (embedding <=> :queryEmbedding)) >= :similarityThreshold
ORDER BY embedding <=> :queryEmbedding
LIMIT :topK

<=> is pgvector’s cosine-distance operator; note the query orders by raw distance (ascending = closer) but the returned similarity score is 1 - distance (so higher = more similar), a translation PgVectorStore performs so your application code always works with “higher = better” semantics regardless of the underlying distance metric a given backend natively uses (some backends natively return distance, not similarity — this normalization is another example of the portability layer doing real work).

HNSW vs. IVFFlat index choice (pgvector-specific but conceptually applicable everywhere): HNSW (Hierarchical Navigable Small World) gives better recall/latency at the cost of slower index builds and higher memory; IVFFlat builds faster and uses less memory but needs a lists parameter tuned to your data size and generally has lower recall at equivalent search-time cost. For most production RAG workloads under a few million vectors, HNSW is the safer default — the recall difference matters more to answer quality than the build-time cost matters to your ingestion pipeline’s SLA.

---

6.5 Metadata Filtering — The Filter.Expression DSL

Filter.Expression filter = new FilterExpressionBuilder()
        .and(
                new FilterExpressionBuilder().eq("category", "returns"),
                new FilterExpressionBuilder().in("region", List.of("IN", "US")),
                new FilterExpressionBuilder().gte("effectiveDate", "2026-01-01")
        )
        .build();

SearchRequest request = SearchRequest.builder()
        .query(userQuery)
        .filterExpression(filter)
        .topK(5)
        .build();

Or the string DSL shortcut for simpler cases (parsed into the same Filter.Expression tree internally):

.filterExpression("category == 'returns' && region in ['IN', 'US']")

Each VectorStore implementation translates this generic Filter.Expression AST into its native filter syntax — SQL WHERE/JSONB containment for pgvector, Pinecone’s metadata filter JSON, RediSearch query syntax, and so on. This is the same portability pattern as ChatOptions: write the filter once, run it against any backend, with the same leaky-abstraction caveat — obscure backend-specific filter capabilities (geo-radius queries on Elasticsearch, for instance) aren’t expressible through the generic DSL and require dropping to backend-specific APIs.

Production-critical use case: multi-tenant isolation. Every similaritySearch call in a multi-tenant RAG system must include a tenantId filter — this is not optional hardening, it’s the primary defense against cross-tenant data leakage through retrieval, and it belongs in a wrapping service/advisor layer that makes it structurally impossible to forget, not something each call site remembers to add:

@Component
public class TenantScopedVectorStore {

    private final VectorStore delegate;

    public List<Document> search(String query, String tenantId, int topK) {
        Filter.Expression tenantFilter = new FilterExpressionBuilder()
                .eq("tenantId", tenantId).build();

        return delegate.similaritySearch(SearchRequest.builder()
                .query(query)
                .topK(topK)
                .filterExpression(tenantFilter)
                .build());
    }
}

---

6.6 Hybrid Search

Pure vector similarity search misses exact-match cases (product SKUs, error codes, proper nouns) that keyword/BM25 search handles naturally — hybrid search combines both, typically via reciprocal rank fusion (RRF) or a weighted score blend.

Beginner note: BM25 is a classic, well-established keyword-ranking algorithm (the same family of technique traditional search engines use) — it’s good at exact and near-exact term matches, precisely where pure vector similarity can be weaker (e.g., a query containing an exact product SKU like "SKU-88213" may not embed distinctively enough for vector search alone to rank it correctly).

Spring AI doesn’t impose one hybrid-search API across all stores because the underlying capability varies too much — instead, backends with native hybrid support (Weaviate, Elasticsearch, Redis via RediSearch) expose it through backend-specific options objects:

// Elasticsearch example — hybrid via combining kNN + BM25 in the underlying query
ElasticsearchVectorStoreOptions options = ElasticsearchVectorStoreOptions.builder()
        .similarity(SimilarityFunction.COSINE)
        // hybrid weighting configured at the query template level for
        // this backend — check the specific version's options surface,
        // this is one of the fastest-moving parts of the vector store API
        .build();

For stores without native hybrid support (plain pgvector without a full-text extension wired in), the production pattern is application-level fusion: run a vector search and a keyword search (e.g., Postgres tsvector full-text query) independently, then merge results with RRF:

public List<Document> hybridSearch(String query, int topK) {
    List<Document> vectorResults = vectorStore.similaritySearch(
            SearchRequest.builder().query(query).topK(topK * 2).build());
    List<Document> keywordResults = fullTextSearchRepository.search(query, topK * 2);

    return reciprocalRankFusion(vectorResults, keywordResults, topK);
}

private List<Document> reciprocalRankFusion(
        List<Document> listA, List<Document> listB, int topK) {
    Map<String, Double> scores = new HashMap<>();
    int k = 60; // RRF constant, standard default from the original paper
    for (int rank = 0; rank < listA.size(); rank++) {
        scores.merge(listA.get(rank).getId(), 1.0 / (k + rank + 1), Double::sum);
    }
    for (int rank = 0; rank < listB.size(); rank++) {
        scores.merge(listB.get(rank).getId(), 1.0 / (k + rank + 1), Double::sum);
    }
    Map<String, Document> byId = Stream.concat(listA.stream(), listB.stream())
            .collect(toMap(Document::getId, d -> d, (a, b) -> a));
    return scores.entrySet().stream()
            .sorted(Map.Entry.<String, Double>comparingByValue().reversed())
            .limit(topK)
            .map(e -> byId.get(e.getKey()))
            .toList();
}

---

6.7 Performance and Production Scaling

Concern	Guidance
Index type	HNSW for most production RAG under a few million vectors; evaluate IVFFlat only if ingestion-time index-build cost is the binding constraint
topK sizing	Retrieve more than you’ll ultimately use (e.g., topK=20) and re-rank down to 3-5 with a cross-encoder or LLM-based reranker (Section 7) — raw vector similarity rank alone is a weak final-relevance signal
Similarity threshold	Set a floor (similarityThreshold) to avoid injecting irrelevant context when no good match exists — returning low-similarity results as if they were relevant is a direct hallucination-risk contributor
Metadata indexing	Ensure your backend indexes filtered metadata fields (e.g., a Postgres GIN index on the jsonb metadata column, or explicit payload indexes in Qdrant) — unindexed metadata filtering degrades to a full scan even with a fast vector index
Sharding/scaling	Vector-native stores (Milvus, Qdrant, Pinecone) handle horizontal sharding for you at scale; pgvector scaling beyond single-node requires the same read-replica/partitioning strategies you’d already use for any large Postgres table
Write throughput	Batch add() calls (same batching principle as Section 5’s embedding guidance) — most VectorStore implementations accept a List<Document> precisely so backend-specific bulk-write APIs get used instead of N individual inserts

---

6.8 Common Mistakes

1. Forgetting a tenant/access-control filter on every query — the highest-severity mistake in this section; treat it as a security control, not an optimization.
2. No similarity threshold, so irrelevant low-similarity documents get injected as “context” and directly increase hallucination risk.
3. Choosing a vector-native database for a small-scale system that already runs Postgres/Redis/Elasticsearch — unnecessary operational surface area for workloads well within what the existing infrastructure handles.
4. Not indexing filtered metadata fields, silently degrading filtered queries to full scans as data grows.
5. Treating raw vector similarity rank as final relevance — skip re-ranking (Section 7) and retrieval quality plateaus well below what’s achievable.
6. Calling add() one document at a time in bulk ingestion instead of batching — mirrors Section 5’s embedding mistake, compounded here since the vector store write is a second network round-trip per item.

---

6.9 Debugging

logging:
  level:
    org.springframework.ai.vectorstore: DEBUG

For pgvector specifically, EXPLAIN ANALYZE the generated SQL directly against Postgres to confirm the HNSW/IVFFlat index is actually being used (a missed index — wrong distance operator, missing USING hnsw on the index definition — silently falls back to a sequential scan, which still returns correct results but with degraded latency that only shows up under load, not in development testing with small datasets).

---

6.10 Interview Questions

1. Why does VectorStore.add() internally call EmbeddingModel, rather than requiring callers to pre-compute embeddings themselves?
2. What’s the practical decision criterion for choosing pgvector vs. a vector-native database like Qdrant or Milvus?
3. Walk through how Filter.Expression gets translated into backend-specific query syntax for two different VectorStore implementations.
4. Why is tenant-scoped filtering described as a security control rather than an optimization, and how would you structurally enforce it across all call sites?
5. Explain the HNSW vs. IVFFlat trade-off and when you’d choose one over the other.
6. What does reciprocal rank fusion do, and when would you need to implement hybrid search at the application level instead of relying on native backend support?
7. Why does raw vector similarity rank make a weak final relevance signal, and what’s the standard production remedy?
8. What happens if you filter on an unindexed metadata field at scale, and how would you detect this in production before it becomes a customer-facing latency problem?
9. Why does PgVectorStore return 1 - distance as similarity instead of the raw pgvector <=> operator output?
10. What’s the operational argument for choosing a vector store you already run in production over one with better benchmark numbers?
11. How would you design a SearchRequest to over-fetch for reranking versus a simple top-K-and-done retrieval?
12. What’s the risk of omitting a similarityThreshold, concretely in terms of downstream model behavior?
13. Describe the schema Spring AI’s PgVectorStore auto-creates, and why initializeSchema(true) is inappropriate for production.
14. How would you batch add() calls for a 100,000-document ingestion job, and what’s the parallel to Section 5’s embedding batching guidance?
15. What’s the difference between metadata filtering capabilities across Qdrant, Pinecone, and plain pgvector, and how does that affect portability of filter-heavy application code?
16. How would you verify that a similarity query is actually using a vector index rather than falling back to sequential scan?
17. What’s the trade-off of retrieving topK=20 and reranking down to 5, versus directly retrieving topK=5?
18. Why might Elasticsearch be an attractive vector store choice specifically for teams already using it for log/search infrastructure?
19. How does horizontal scaling differ between a vector-native database and pgvector at very large (billion-vector) scale?
20. What’s the correct production alternative to initializeSchema(true) for schema management?

---

6.11 Best Practices Checklist

• Structurally enforce tenant/access-control filtering on every retrieval call — never leave it to individual call sites to remember.
• Always set a similarityThreshold floor to avoid injecting irrelevant context.
• Default to your already-operated infrastructure (Postgres/Redis/Elasticsearch) unless scale really demands a vector-native database.
• Index filtered metadata fields explicitly; verify with EXPLAIN ANALYZE or the backend equivalent.
• Over-fetch and rerank rather than trusting raw similarity rank as final relevance (bridges into Section 7).
• Batch add() calls for bulk ingestion; never loop one document at a time.
• Manage schema via Flyway/Liquibase in production, not initializeSchema(true).

---

6.12 Key Takeaways

• VectorStore unifies storage + embedding + retrieval behind one contract — it owns the embedding step, not just persistence.
• The “best” vector store is usually the one your team already operates, not the one with the best raw benchmark.
• Metadata filtering is both a relevance tool and, in multi-tenant systems, a mandatory security control.
• Hybrid search availability varies significantly by backend — application-level RRF fusion is the portable fallback.
• Raw similarity rank is a weak final-relevance signal; production RAG pipelines over-fetch and rerank (Section 7 builds directly on this).

---

End of Section 6. Next: Section 7 — RAG in Spring AI (Retriever, Advisor, Document Readers, Chunking, Tokenization, Embedding Pipeline, Ranking, Context Building, Production Pipeline, Evaluation, Hallucination Reduction).