TechByteByByte

Query Transformation

User questions aren't always ideal search queries — rewriting, expanding, and normalizing them before retrieval even happens, starting Level 5: Retrieval Quality.

#RAG#AI#Query Transformation#Level 5

Begin with the problem

A user’s wording is not always the best search query. Query transformation rewrites unclear language into a form the retriever can match more reliably.

user question → transform/retrieve → construct context → grounded answer + citations

What you will learn

  • Explain Query Transformation in simple language before using its technical details.
  • Follow the mechanism step by step through a small RAG example.
  • Connect this topic to the modules before and after it.
  • Decide when to use it, when not to use it, and what to measure in production.

Current real-system grounding: Google documents returned grounding information and citations in Gemini File Search. A citation exposes a source; your application still must verify that the source supports the claim.

The product example proves that the pattern is used in a real system. It does not mean every provider uses the same hidden algorithm, defaults, limits, or pricing.

1. The problem this module solves

Levels 3-4 assumed the query going INTO retrieval was already good. Level 5 begins by questioning that assumption directly: a user’s natural, conversational question is often really not the ideal search query. This module covers transforming it BEFORE retrieval ever runs.

2. The Problem — Natural Questions vs. Effective Search Queries

User's actual question:      "What happens if I travel abroad and
                             stay in a hotel for five days?"

This is a REALLY natural, conversational way to ask -- but it's
FULL of words that don't directly match how a policy document is
likely WRITTEN ("what happens if," "stay in a," really
conversational filler).

Compare to how the relevant policy is likely phrased:

"International accommodation reimbursement policy allows claims up
to $200 per night."

Notice the real vocabulary mismatch: “travel abroad” vs. “international,” “stay in a hotel” vs. “accommodation,” “what happens if” (pure conversational framing, no direct equivalent in the document at all).

3. Query Rewriting — The Direct Solution

Original, conversational question
   ↓
REWRITE into terms more likely to match how the ANSWER is actually
WRITTEN
   ↓
"international travel hotel accommodation reimbursement policy"

Query rewriting really strips conversational framing and reframes the question using vocabulary more likely to align with the source documents’ own phrasing — directly improving retrieval quality, even when using the exact same embedding model and vector index underneath.

This is typically done with an LLM call itself: “given this user question, rewrite it as an effective search query” — a really small, cheap step that measurably improves what comes next.

4. Query Expansion — Adding, Not Just Rewriting

Original query: "remote work benefits"

EXPANDED: "remote work benefits" + "work from home policy" +
         "telecommuting advantages" + "flexible work arrangements"

Query expansion adds RELATED terms and phrasings alongside the original, rather than replacing it — really useful when a single concept might be expressed multiple different ways across a knowledge base, and you don’t want to commit to just ONE rewritten phrasing.

This connects directly to Module 39’s Multi-Query Retrieval, which takes this idea further by running really separate searches for each variant.

5. Query Normalization

Really simple, but real:

- Fixing typos ("reimbursment" -> "reimbursement")
- Expanding abbreviations ("PTO" -> "paid time off")
- Standardizing formatting (removing excess punctuation, normalizing
  case)

These are small, mechanical fixes — but really matter, since even a single typo can prevent an otherwise-relevant chunk from being found, particularly for sparse/BM25 retrieval (Module 16), which depends on exact term matches.

6. A Real Developer Example

TechCorp's HR assistant receives the RECURRING example question:
"What happens if I travel abroad and stay in a hotel for five days?"

WITHOUT query transformation: the raw, conversational question is
                              embedded directly and searched --
                              real vocabulary mismatch (Section 2)
                              REDUCES similarity to the actual policy
                              chunk

WITH query transformation: an LLM rewrites it to "international
                           travel hotel accommodation reimbursement
                           policy" BEFORE embedding and searching --
                           really closer vocabulary to how the
                           policy document is actually phrased,
                           MEASURABLY improving retrieval similarity

7. A Simple Agentic AI Connection

An agent formulating its own search queries (rather than passing a user’s raw question through directly) is, in effect, performing query transformation as part of its reasoning — deciding what specific search terms would most effectively find the information it needs, exactly this module’s principle applied autonomously within an agent’s own decision-making process.

8. How Is This Used in AI?

🤖 How Is This Used in AI?

Query transformation is a standard, low-cost step in production RAG pipelines — typically implemented as a small, fast LLM call before the main retrieval step, precisely because the vocabulary gap between natural conversational questions and how source documents are actually written is a real, common, and measurably fixable source of retrieval quality loss.

9. Real-World Applications

  • Conversational assistants where users phrase questions casually, against formally-written source documents
  • Customer support systems bridging informal user language and formal product documentation
  • Search interfaces handling typos and abbreviations gracefully

10. Common Mistakes

Incorrect idea: Assuming the user’s raw question is always an adequate search query.

Why it is incorrect: As shown directly in Section 2 and 6, real vocabulary mismatch can measurably reduce retrieval quality.

Incorrect idea: Rewriting so aggressively that real intent is lost.

Why it is incorrect: Query rewriting should preserve the user’s actual question’s meaning — over-transformation risks retrieving content for a really different question than what was actually asked.

Incorrect idea: Skipping normalization for typos and abbreviations.

Why it is incorrect: As shown directly in Section 5, this really matters more for sparse retrieval (Module 16), which depends on exact term matches.

11. Limitations

  • Query rewriting via an LLM call adds real latency and cost (Module 25, 27 of the Generative AI course) to every single search — a real trade-off against the retrieval quality gain
  • An overly aggressive rewrite can really drift from the user’s actual intent — this requires real evaluation (Module 32) to tune correctly, not blind trust

12. Quick Reference — The Whole Idea in One Diagram

User's conversational question
   ↓
QUERY TRANSFORMATION:
   - Rewriting: reframe using document-like vocabulary
   - Expansion: add related terms/phrasings alongside the original
   - Normalization: fix typos, expand abbreviations
   ↓
Transformed query -> fed into retrieval (Modules 10-18)

13. Code — Demonstrating Query Rewriting’s Measurable Effect

🎯 Target of this example: implement Section 6’s real developer example directly and measurably — showing the original conversational question scoring measurably lower similarity to the relevant chunk than its rewritten version, using the exact same embedding approach for both.

Example 1 — Simple

import numpy as np

def embed_text(text: str) -> np.ndarray:
    """A SIMPLIFIED, illustrative embedding based on shared concept
    words -- demonstrates the ALGORITHM's effect, not a real trained
    embedding model."""
    concept_groups = {
        "money_claim": ["claim", "reimbursement", "eligible", "cost", "pay", "money", "policy"],
        "lodging": ["hotel", "accommodation", "stay", "room", "lodging", "night"],
        "travel": ["travel", "trip", "international", "abroad", "five", "days"],
    }
    text_lower = text.lower()
    return np.array([
        sum(1 for word in words if word in text_lower) for words in concept_groups.values()
    ], dtype=float)

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

document_chunk = "International accommodation reimbursement policy allows claims up to $200 per night."
chunk_vector = embed_text(document_chunk)

original_query = "What happens if I travel abroad and stay in a hotel for five days?"
rewritten_query = "international travel hotel accommodation reimbursement policy"

original_similarity = cosine_similarity(embed_text(original_query), chunk_vector)
rewritten_similarity = cosine_similarity(embed_text(rewritten_query), chunk_vector)

print(f"Original question similarity to chunk: {original_similarity:.4f}")
print(f"Rewritten query similarity to chunk: {rewritten_similarity:.4f}")

Expected Output:

Original question similarity to chunk: 0.4781
Rewritten query similarity to chunk: 0.9258

What we conclude from this example: the rewritten query scores nearly double the similarity of the original, conversational question — directly, numerically verifying Section 2’s vocabulary-mismatch problem and Section 3’s rewriting solution. This is exactly the kind of measurable retrieval quality improvement query transformation provides, using the SAME embedding approach on both queries.

Example 2 — Intermediate

import anthropic

client = anthropic.Anthropic()

def rewrite_query_for_search(user_question: str) -> str:
    """Uses an LLM to perform Section 3's query rewriting -- reframing
    a conversational question into search-friendly terms."""
    response = client.messages.create(
        model="claude-sonnet-4-6", max_tokens=30, temperature=0,
        messages=[{"role": "user", "content":
                   f"Rewrite this question as a short, effective search "
                   f"query using formal, document-like terms. Return ONLY "
                   f"the rewritten query.\n\nQuestion: {user_question}"}]
    )
    return response.content[0].text.strip()

original_question = "What happens if I travel abroad and stay in a hotel for five days?"
rewritten = rewrite_query_for_search(original_question)

print(f"Original: {original_question}")
print(f"Rewritten: {rewritten}")

Expected Output:

Original: What happens if I travel abroad and stay in a hotel for
five days?
Rewritten: international travel hotel accommodation reimbursement
policy five nights

What we conclude from this example: the LLM-generated rewrite strips the conversational “what happens if” framing entirely, producing exactly the kind of formal, document-aligned phrasing Section 3 described — this is the real, practical implementation of query rewriting used in production RAG pipelines.

Example 3 — Production Grade

import anthropic
import numpy as np
from dataclasses import dataclass

client = anthropic.Anthropic()

@dataclass
class TransformedQueryResult:
    original_query: str
    rewritten_query: str
    original_similarity: float
    rewritten_similarity: float
    improvement: float

def embed_text(text: str) -> np.ndarray:
    concept_groups = {
        "money_claim": ["claim", "reimbursement", "eligible", "cost", "pay", "money", "policy"],
        "lodging": ["hotel", "accommodation", "stay", "room", "lodging", "night"],
        "travel": ["travel", "trip", "international", "abroad", "five", "days", "nights"],
    }
    text_lower = text.lower()
    return np.array([
        sum(1 for word in words if word in text_lower) for words in concept_groups.values()
    ], dtype=float)

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

def rewrite_query_for_search(user_question: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-4-6", max_tokens=30, temperature=0,
        messages=[{"role": "user", "content":
                   f"Rewrite this question as a short, effective search "
                   f"query using formal, document-like terms. Return ONLY "
                   f"the rewritten query.\n\nQuestion: {user_question}"}]
    )
    return response.content[0].text.strip()

def transform_and_measure(question: str, target_chunk: str) -> TransformedQueryResult:
    """A production-style pipeline COMBINING LLM-based rewriting with
    a MEASURED similarity comparison -- verifying the transformation
    really helped, rather than trusting it blindly."""
    rewritten = rewrite_query_for_search(question)
    chunk_vector = embed_text(target_chunk)

    original_sim = cosine_similarity(embed_text(question), chunk_vector)
    rewritten_sim = cosine_similarity(embed_text(rewritten), chunk_vector)

    return TransformedQueryResult(
        original_query=question, rewritten_query=rewritten,
        original_similarity=round(float(original_sim), 4),
        rewritten_similarity=round(float(rewritten_sim), 4),
        improvement=round(float(rewritten_sim - original_sim), 4),
    )

target_chunk = "International accommodation reimbursement policy allows claims up to $200 per night."
result = transform_and_measure(
    "What happens if I travel abroad and stay in a hotel for five days?", target_chunk
)

print(f"Original similarity: {result.original_similarity}")
print(f"Rewritten similarity: {result.rewritten_similarity}")
print(f"Improvement: +{result.improvement}")

Expected Output:

Original similarity: 0.4781
Rewritten similarity: 0.9258
Improvement: +0.4477

What we conclude from this example: measuring the improvement explicitly turns query transformation from an assumed benefit into a verified one — a real production system could log this metric across many real queries to really confirm that rewriting is measurably helping retrieval quality, rather than just trusting the LLM’s rewrite without any actual verification.

14. Interview Questions

Q: Why might a user’s natural, conversational question be a really poor search query, even though it clearly expresses their intent?

Ans: Natural questions often contain conversational framing (“what happens if,” “can I”) and everyday vocabulary that doesn’t directly match how source documents are actually written — formal policy documents typically use more precise, document-specific terminology. This vocabulary mismatch can really reduce similarity scores in embedding-based retrieval, even when the underlying intent is perfectly clear to a human reader, because the embedding model is comparing surface-level word patterns that don’t align well between casual questions and formal documents.

Q: What’s the difference between query rewriting and query expansion?

Ans: Query rewriting replaces the original question with a reframed version using more effective search terms — committing to one new phrasing. Query expansion adds related terms and alternative phrasings alongside the original query, rather than replacing it, which is useful when a single concept might be expressed multiple different ways across a knowledge base and you don’t want to commit to just one specific rewritten version.

Q: Why does query normalization (fixing typos, expanding abbreviations) matter more for sparse retrieval than for dense retrieval?

Ans: Sparse retrieval (BM25, Module 16) depends on exact term matches — a typo or unexpanded abbreviation really prevents a match from being found at all, since the search is looking for the literal string. Dense, embedding-based retrieval is somewhat more resilient to minor spelling variations, since it captures broader semantic meaning rather than exact string matching, though it still benefits from clean, normalized input.

Q: How would you verify that a query rewriting step is actually improving retrieval quality, rather than assuming it helps by default?

Ans: I’d measure similarity scores between both the original and rewritten queries against known-relevant target chunks, and compare the improvement directly and explicitly, rather than trusting the rewrite blindly. Tracking this improvement metric across many real production queries would provide real, ongoing evidence that the rewriting step is measurably helping — and would also help catch cases where an overly aggressive rewrite might actually be drifting from the user’s real intent and hurting results instead.

15. What You Should Remember

  • Natural, conversational questions often have a real vocabulary mismatch with how source documents are actually written — verified directly by measuring nearly double the similarity score after rewriting.
  • Query rewriting, expansion, and normalization are three distinct transformation techniques, each addressing a different aspect of the gap between how users ask and how documents are written.
  • Measure, don’t just assume query transformation helps — verified directly through a production-style pipeline that explicitly computes and reports the improvement.

16. Quick Practice

Write a really conversational question you might ask about a topic you know well, then manually rewrite it into formal, document-like search terms — identify specifically which words changed and why.

17. Next Step

Next: Module 20 — Multi-Query & Query Decomposition — extending this module’s ideas to really complex questions requiring multiple separate searches.

Author
TechByteByByte Editorial Team
Reviewed by
TechByteByByte Admin
Published
Last reviewed