Query, Key and Value | TechByteByByte

Begin with the central question

Why does attention create three different vectors from each token instead of using one vector for everything?

Essential words

A Query represents what a position seeks. A Key represents what a position offers for matching. A Value carries the information collected after matching.

What You Will Understand

Why real self-attention uses three separate learned projections of the same input — Query, Key, and Value — rather than the raw embeddings directly (as Module 3’s simplified version did), with a complete, verified numeric example showing each one is a genuinely different vector, even for the same token.

token state -> Q, K, V projections -> matches -> collected values

The problem this module solves

Module 3 built attention using raw embeddings for everything: the same vector was used to measure relevance and to be the actual content retrieved. This works for building intuition, but it’s a real limitation — it forces a token’s “how relevant am I to others” role and its “what information do I actually contribute” role to be the same vector, when these are conceptually different jobs.

Q/K/V exists to give each job its own dedicated, learnable representation.

Build the intuition

imagine a library search. Your search terms are the Query (“what am I looking for?”). Each book has a set of index tags describing its contents — that’s the Key (“what do I contain, for matching purposes?”). But once you’ve found the right book, what you actually read is its full text — the Value (“the actual information I provide, once selected”). A book’s index tags and its full text serve different purposes, even though they both describe the same book — exactly why Key and Value are separate.

4. Real-World Analogy

Think of an online marketplace search. You type a query. Every product listing has searchable metadata (tags, title, category) — its Key. But what you actually receive after clicking through and buying is the full product — its Value.

The metadata that made it findable isn’t necessarily identical to everything the product actually is — two different representations of the same underlying item, serving two different purposes.

Analogy: The Job Recruitment Marketplace (Resume, Job Requirements, Work Output) Imagine a technical job recruitment site where candidate profiles are matched:

The Search Resume (Query - Q): A software engineer updates their search bio: “Senior Java developer looking for a database tuning role.” (What am I searching for?)

The Job Requirements (Key - K): A company lists tags for a database role: “Required: Java, SQL index optimization.” (What tags do I offer for matching?)

The Code Delivery (Value - V): The engineer’s actual work output, systems delivery, and code quality once hired (the actual information/content contributed).

The Matching: The system matches resume text (Q) against job tags (K) to get a compatibility score (Attention weight). But the company doesn’t hire the resume; it hires the engineer’s actual coding skills (V).

By projecting one base profile (token) into three separate DTO roles, the candidate is matched efficiently without compromising their actual work output.

📊 Visual Flowchart: Query-Key-Value Linear Projection Stack

Here is how input matrix X is projected into distinct Q, K, and V spaces using separate learnable parameters:

graph TD
    X["Input Embedding X<br>(Seq Length x d_model)"] --> QProj["1. Query Projection<br>(Q = X @ W_Q)"]
    X --> KProj["2. Key Projection<br>(K = X @ W_K)"]
    X --> VProj["3. Value Projection<br>(V = X @ W_V)"]




    QProj --> QMatrix["Query Matrix Q<br>(Seq Length x d_k)"]
    KProj --> KMatrix["Key Matrix K<br>(Seq Length x d_k)"]
    VProj --> VMatrix["Value Matrix V<br>(Seq Length x d_v)"]




    QMatrix --> ScoreMatrix["4. Similarity Product<br>(Q @ K.T)"]
    KMatrix --> ScoreMatrix




    ScoreMatrix --> Softmax["5. Softmax Weights"]




    subgraph Blending ["Retrieve and Synthesize"]
        Softmax --> finalBlend["6. Attention Output<br>(weights @ V)"]
        VMatrix --> finalBlend
    end

5. Core Concept

X                     (the same input embedding — Module 2's output)
  ↓
   ├──→ W_Q →  Q     ("what am I looking for?")
   ├──→ W_K →  K     ("what do I offer, for matching purposes?")
   └──→ W_V →  V     ("what information do I actually provide?")

Each of W_Q, W_K, W_V is a separate, independently learned weight matrix (exactly Deep Learning Module 2’s linear projection, applied three times with three different weight sets). All three start from the same input X, but produce three genuinely different vectors per token.

Why Q interacts with K, and V holds the retrieved information

Q × Kᵀ
  ↓
Similarity scores      (how well does each token's Query match
                        every token's Key?)
  ↓
Softmax
  ↓
Attention weights
  ↓
× V
  ↓
Contextual representation   (a weighted blend of VALUES, weighted
                            by Query-Key relevance)

🧠 Notice: Query and Key are only ever used to compute relevance scores — they never directly appear in the final output. Value is the only one of the three that actually contributes content to the final weighted sum. This division of labor is precisely why three separate projections are more expressive than reusing one vector for everything.

6. How It Works — Step by Step

1. Take the input embeddings X (Module 2)
2. Compute Q = X @ W_Q  -- a learned projection, producing each
   token's "what am I looking for" vector
3. Compute K = X @ W_K  -- a SEPARATE learned projection,
   producing each token's "what do I offer" vector
4. Compute V = X @ W_V  -- a THIRD separate learned projection,
   producing each token's "what information do I contribute" vector
5. Compute similarity scores: Q @ K^T (every token's Query
   compared against every token's Key)
6. Softmax the scores -> attention weights
7. Compute the weighted sum of V (NOT of Q or K) using these
   weights -- this is the final contextual output

7. Mathematical Intuition

Read the mathematics as a story

One token needs separate representations for searching, being matched, and supplying information. Matrix multiplication creates those roles from the same starting state.

X @ WQ = Q: what I seek
X @ WK = K: how I can be matched
X @ WV = V: information I contribute

W_Q, W_K, and W_V can each have their own output dimension (often called d_k for Q/K, d_v for V) — not necessarily the same as the input embedding dimension d_model. This means the projections can also compress or reshape the representation, not just rotate it — demonstrated directly below, where a 4-dimensional input embedding becomes a 3-dimensional Query, Key, and Value.

8. Small Worked Example

Walk through the example

Start with one four-number token vector. 2. Apply three projection matrices. 3. Compare the resulting Q, K, and V. 4. Use Q and K for matching, then mix V.

For the token “tired” (from Module 3’s sentence), its raw embedding is one specific 4-dimensional vector. After applying three separate projections, “tired” now has three different vectors: a Query vector (for searching), a Key vector (for being found), and a Value vector (for contributing content) — verified to be genuinely different from one another below, not just relabeled copies of the same numbers.

9. Python / NumPy Example

What the code will demonstrate

This small NumPy example makes Query, Key and Value visible with inspectable numbers and shapes. Read it in three passes: identify each input, follow the transformation line by line, and connect the printed output to the diagram above. The arrays are intentionally tiny teaching values; unless the text explicitly says otherwise, they are not weights or measurements from GPT, Gemini, or another trained model.

# The arrays are intentionally small so each transformation can be inspected.
# Printed values illustrate the mechanism; they are not trained-model measurements.
import numpy as np




def softmax_rows(x):
    exp_x = np.exp(x - np.max(x, axis=-1, keepdims=True))
    return exp_x / np.sum(exp_x, axis=-1, keepdims=True)




tokens = ["animal", "was", "tired"]
X = np.array([
    [0.9, 0.8, 0.1, 0.2],
    [0.1, 0.0, 0.9, 0.1],
    [0.8, 0.7, 0.2, 0.9],
])
print("Input embeddings X:\n", X)




d_model = 4
d_k = 3   # deliberately SMALLER than d_model, to show projections
          # can change dimensionality, not just rotate




np.random.seed(1)
W_Q = np.round(np.random.randn(d_model, d_k) * 0.5, 2)
W_K = np.round(np.random.randn(d_model, d_k) * 0.5, 2)
W_V = np.round(np.random.randn(d_model, d_k) * 0.5, 2)




Q = X @ W_Q
K = X @ W_K
V = X @ W_V




print("\nQuery matrix Q:\n", np.round(Q, 3))
print("Shape:", Q.shape, "-- notice d_model=4 became d_k=3")
print("\nKey matrix K:\n", np.round(K, 3))
print("\nValue matrix V:\n", np.round(V, 3))




# Same token, three different roles
print("\n--- Same token ('tired'), three different roles ---")
print("Original embedding: ", X[2])
print("Query vector:        ", np.round(Q[2], 3))
print("Key vector:          ", np.round(K[2], 3))
print("Value vector:        ", np.round(V[2], 3))
print("Q == K?", np.array_equal(Q[2], K[2]))
print("K == V?", np.array_equal(K[2], V[2]))




# Full attention computation using Q/K/V
scores = Q @ K.T
scaled_scores = scores / np.sqrt(d_k)
weights = softmax_rows(scaled_scores)
print("\nAttention weights:\n", np.round(weights, 4))




output = weights @ V
print("\nAttention output (weights @ V):\n", np.round(output, 4))
print("Shape:", output.shape)

Expected Output:

Input embeddings X:
 [[0.9 0.8 0.1 0.2]
 [0.1 0.  0.9 0.1]
 [0.8 0.7 0.2 0.9]]




Query matrix Q:
 [[ 0.36   0.173 -1.344]
 [ 0.852 -0.3    0.015]
 [ 0.336  0.634 -1.908]]
Shape: (3, 3) -- notice d_model=4 became d_k=3




Key matrix K:
 [[-0.468 -0.124  0.156]
 [ 0.059  0.287 -0.413]
 [ 0.004  0.248  0.263]]




Value matrix V:
 [[-0.09  -0.498  0.126]
 [-0.312 -0.248 -0.313]
 [-0.417 -0.709  0.064]]




--- Same token ('tired'), three different roles ---
Original embedding:  [0.8 0.7 0.2 0.9]
Query vector:         [ 0.336  0.634 -1.908]
Key vector:           [0.004 0.248 0.263]
Value vector:         [-0.417 -0.709  0.064]
Q == K? False
K == V? False




Attention weights:
 [[0.259  0.4682 0.2729]
 [0.2954 0.3548 0.3497]
 [0.221  0.5324 0.2466]]




Attention output (weights @ V):
 [[-0.2832 -0.4385 -0.0964]
 [-0.2831 -0.4831 -0.0514]
 [-0.2888 -0.4169 -0.123 ]]
Shape: (3, 3)

How It Works

Q == K? False and K == V? False, confirmed directly — even for the exact same token, the three projections produce genuinely distinct vectors. Nothing about this is redundant relabeling: W_Q, W_K, and W_V are three independently-learned weight matrices, each free to emphasize different aspects of the original embedding for its specific purpose.
The output shape (3, 3) matches V’s dimension (d_k=3), not the original d_model=4 — proof that Q/K/V projections can genuinely reshape the representation, and that the final attention output’s dimensionality is determined by V’s projection specifically, not by the original embedding size.
Notice the final output uses V exclusively (weights @ V) — Q and K were used entirely for computing the weights, then never touched again. This is the concrete confirmation of Section 5’s division of labor.

10. How Is This Used in Modern AI?

Where this concept lives

Follow the concept at three levels: inside the model, where the computation happens; inside the AI product, where that computation supports a visible feature; and inside production, where engineers measure speed, memory, quality, and failure cases. The details below connect those levels.

🤖 How Is This Used in Modern AI?

Every attention computation in every modern Transformer — including every layer of every LLM — uses this exact three-projection structure. W_Q, W_K, and W_V are genuine, independently-learned parameters (Deep Learning Module 2, 7-9), adjusted during training just like any other weight matrix, at every attention layer.

11. How Is This Used in LLMs?

Trace one model call

User text → tokens → Transformer computation → output-token probabilities
          this topic affects one part of that computation

An LLM does not apply this idea as a separate magic step. It uses it as part of the repeated numerical pipeline that transforms token vectors and produces the next-token probabilities.

In a real LLM, W_Q, W_K, and W_V are learned during pretraining to capture genuinely useful patterns — for instance, a token’s Key vector might come to encode “I am a noun that could be a pronoun’s antecedent,” while a later pronoun’s Query vector encodes “I’m looking for my antecedent.” Neither of these specific interpretations is guaranteed or hand-designed — they’re simply patterns that emerge if they help the model perform well on its training objective (next-token prediction, Module 15 of this course).

Real systems you can recognize

Hugging Face describes KV caches as storing per-layer key and value tensors during autoregressive generation; see caching. The query for the new token is matched against current and cached keys.

12. How Is This Used in Agentic AI?

Trace one agent step

Goal + history + tool results
              ↓
     LLM processes the context
              ↓
Suggested answer or tool call
              ↓
Agent runtime validates and executes it

This distinction matters: the Transformer helps produce the proposal, while the surrounding agent software controls tools, permissions, retries, memory, and execution.

Direct relevance to Agentic AI: High, indirectly. Every contextual connection an agent’s LLM makes — relating a new instruction to earlier conversation, or connecting a tool result back to the original request — runs through exactly this Q/K/V mechanism, at every attention layer of the underlying model.

When this knowledge is useful

Use Query, Key and Value when you need to explain, implement, debug, evaluate, or optimize the corresponding part of a Transformer pipeline. It is also useful when a model API behaves unexpectedly and you need to trace the behavior back to tokens, tensor shapes, attention visibility, training, or inference mechanics.

When it is not enough

Understanding this mechanism does not by itself prove that a complete model or application is accurate, safe, fast, or cost-effective. Production decisions still require representative evaluation data, latency and memory measurements, model-specific documentation, and tests of the surrounding retrieval or agent code.

13. Common Beginner Mistakes

⚠️ Mistake

Incorrect idea: assuming Q, K, and V are just three copies of the same vector under different names.

Why it is incorrect: As verified directly (Q == K? False) — they are three genuinely different, independently-learned projections of the same input.

⚠️ Mistake

Incorrect idea: thinking Query and Key must have the same dimension as Value.

Why it is incorrect: They can (and this module deliberately shows an example where all three share d_k), but this isn’t required — Q and K only need to match each other’s dimension (since they’re compared via Q @ K^T), while V’s dimension determines the final output’s dimension independently.

⚠️ Mistake

Incorrect idea: believing Q and K “do the same thing.”

Why it is incorrect: They serve distinct roles — Q represents “what this token is looking for,” K represents “what this token offers as a potential match” — and are compared against each other, not used interchangeably.

14. Important Distinctions

Q vs K	K vs V
Q = “what am I looking for”	K = “what do I offer, for matching”
Compared against K, via dot product	V = “what I actually contribute,” once matched
Never appears in the final output directly	V is the ONLY one of the three used in the final weighted sum

Single-Vector Attention (Module 3)	Q/K/V Attention (this module)
Uses the same raw embedding for relevance AND content	Uses three separately-learned projections, each specialized
Simpler, good for building intuition	What real Transformers actually use

15. Production / Engineering Considerations

W_Q, W_K, W_V are real, trainable parameters contributing to a model’s total parameter count and memory footprint at every attention layer — for a model with many layers and a large d_model, these matrices collectively represent a meaningful fraction of total model size, directly relevant when reasoning about model size and inference memory (Module 17).

16. Interview Questions

Beginner

Q: What are Query, Key, and Value, in one sentence each?

Ans: Query represents what a token is “looking for” in other tokens. Key represents what a token “offers” as a potential match for other tokens’ queries. Value is the actual information a token contributes to the output, once it’s been selected as relevant via the Query-Key match.

Intermediate

Q: Why does self-attention need three separate projections instead of using the raw embedding directly for everything, as in a simplified version?

Ans: Using the raw embedding for everything forces a token’s “how relevant am I to others” role and its “what content do I actually provide” role to be identical — but these are conceptually different jobs.

Separate learned projections let the model optimize each role independently: W_Q/W_K can specialize in producing good matching signals, while W_V can specialize in producing good content to retrieve, without either being constrained by the other’s requirements.

Advanced

Q: Why can Value have a different dimension than Query and Key, and what does that dimension determine?

Ans: Query and Key must share the same dimension because they’re directly compared via a dot product (Q @ K^T) to compute similarity scores — this operation requires matching dimensions. Value has no such constraint, since it’s only combined via a weighted sum after the attention weights are already computed.

Value’s dimension directly determines the dimension of the final attention output, as demonstrated in this module, where a 4-dimensional input produced a 3-dimensional final output because V’s projection dimension was 3.

Scenario

Q: You inspect a trained Transformer and notice that for a given token, its Query and Key vectors are numerically quite different from each other, even though they came from the same input embedding. Is this expected?

Ans: Yes — this is exactly expected, and demonstrated directly in this module (Q == K? False). W_Q and W_K are independently initialized and independently trained matrices; there’s no requirement or mechanism that would make them converge to producing similar vectors for the same input. Each is free to learn whatever transformation best serves its specific role (searching vs. being found).

Architecture

Q: In the attention computation, which of Q, K, V directly determines the final output’s values, and which only influence the WEIGHTS used to combine them?

Ans: Value directly determines the final output’s content — the output is literally a weighted sum of Value vectors. Query and Key only influence which weights are used in that sum; neither appears in the final weighted-sum computation itself, only in the score/weight calculation that precedes it.

AI Engineering

Q: When debugging unexpected attention behavior in a Transformer-based model, why might it be useful to inspect Q, K, and V separately rather than just the final attention output?

Ans: Since Q and K together determine which tokens get weighted heavily (the attention pattern itself), while V determines what content gets contributed once weighted, separating these lets you diagnose different kinds of issues: if attention weights look reasonable but the final output seems unhelpful, the issue may lie in W_V’s learned representations rather than in the relevance-matching mechanism itself — a distinction the final output alone wouldn’t reveal.

17. What You Should Remember

Query, Key, and Value are three separately-learned projections of the same input embedding — verified directly to be genuinely different vectors, even for the same token.
Q and K are compared (via dot product) to compute relevance; only V contributes to the final output — a clear division of labor.
Projections can change dimensionality, not just rotate — the final attention output’s dimension is determined by V’s projection.

18. How This Helps Me Build AI Systems

You now understand precisely what those three letters — Q, K, V — mean every time you encounter them in a Transformer diagram, paper, or codebase, and you’ve verified with real numbers that they’re genuinely distinct, purpose-built projections, not an arbitrary naming convention. Module 5 formalizes this exact computation into the standard scaled dot-product attention equation.

Next: Module 5 — Scaled Dot-Product Attention — the formal equation, and precisely why the √dₖ scaling factor is necessary.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026