Prompt Optimization | TechByteByByte

Start with the real problem

Prompt optimization means making tested changes that improve a result we care about, such as accuracy, cost, or response time.

Optimization is not making a prompt longer or more impressive. It is finding the smallest tested change that improves a defined outcome without unacceptable cost or regressions.

baseline → eval → diagnose → change one factor → compare → keep or revert

What you will learn

Choose measurable optimization targets.
Protect a baseline and set of tests that catches new breakage.
Optimize prompt, model, context, and parameters separately.
Stop when gains no longer justify complexity.

How this connects to current AI systems

Current provider guidance treats prompt design as iterative and recommends varied, realistic tests rather than universal magic phrases.

1. Why This Module Exists

This module doesn’t introduce a new technique — it’s a synthesis. Every piece is already covered: iteration (Module 14), evaluation (Module 20), versioning (Module 21), token economics (Module 25), and model/ parameter choices (Module 26). This module assembles them into one coherent, repeatable optimization process.

2. The Idea, in Plain Language

Prompt optimization means systematically improving a prompt across multiple dimensions — clarity, reliability, cost, and speed — using the tools this course has already covered, in a deliberate order.

Define task
   ↓
Create baseline (Module 14's first attempt)
   ↓
Create test cases (Module 20)
   ↓
Run prompt against test cases
   ↓
Identify failure patterns (Module 14's diagnosis step)
   ↓
Modify prompt
   ↓
Evaluate again (Module 20)
   ↓
Compare against baseline
   ↓
Version (Module 21)
   ↓
Deploy

3. Why Order Matters

It’s worth being explicit about the sequence here, since optimizing in the wrong order wastes real effort:

1. CORRECTNESS FIRST         -- does the prompt reliably produce the
                              right kind of output at all? (Module
                              2, 3, 8, 9)

2. RELIABILITY SECOND           -- does it work consistently across
                                varied, realistic inputs, not just
                                one example? (Module 14, 20)

3. EFFICIENCY LAST                 -- once it's correct and
                                   reliable, THEN trim unnecessary
                                   length (Module 25) and tune
                                   generation parameters (Module 26)

Optimizing for efficiency before correctness and reliability are established is a common, real mistake — you end up optimizing something that doesn’t actually work yet, or cutting content that turns out to have been really necessary.

4. A Full Worked Optimization Pass

TASK: Extract order numbers from customer messages for a support tool.

STEP 1 - Baseline:
"Extract the order number from this message."
→ Correctness check: works on obvious cases, fails on edge cases
  (missing numbers, unusual formatting)

STEP 2 - Test cases (Module 20):
10 realistic messages, including edge cases, with expected outputs.

STEP 3 - Run baseline against test cases:
6/10 correct -- failures on missing-number and formatting-variation
cases.

STEP 4 - Diagnose specific failures (Module 14):
- Missing numbers: no instruction for what to do when absent
- Formatting variations ("order# 4471", "Order: 4471"): no
  instruction to normalize/strip formatting

STEP 5 - Revise:
"Extract the order number from this message. Return only the numeric
digits, no prefix or symbols. If no order number is mentioned, return
exactly: NOT_FOUND"

STEP 6 - Re-evaluate:
9/10 correct -- one remaining edge case (multiple order numbers
mentioned) identified for a further iteration.

STEP 7 - Version and deploy (Module 21):
This becomes v2, replacing v1, with the specific improvement and
remaining known gap documented.

STEP 8 - Efficiency pass (Module 25):
Prompt is already fairly concise -- no significant padding to trim.
Temperature set to 0 (Module 26) since consistency matters far more
than variety for this extraction task.

Notice: every single step maps directly to a module you’ve already covered — this is really the synthesis this module promised, not a new set of ideas.

5. A Real Example From a Developer’s Perspective

Here’s how a real optimization pass often reveals a genuine trade-off worth making deliberately:

v3 (highly detailed, very reliable): 98% accuracy on test set, but
uses roughly 180 tokens per request.

v4 (trimmed, still reliable): 96% accuracy on the same test set,
uses roughly 90 tokens per request.

Decision: for a high-volume feature (50,000+ requests/month), the 2%
accuracy difference may be an acceptable trade-off for a 50% token
cost reduction -- but this is a genuine business decision, not
something to decide purely on "shorter is better" instinct. The right
choice depends on how costly that remaining 2% of errors actually is
in this specific context.

This is exactly the kind of deliberate, evidence-based trade-off (Module 25) that a systematic optimization process makes possible — rather than either blindly maximizing accuracy regardless of cost, or blindly minimizing cost regardless of accuracy.

6. A Simple Agentic AI Example

Optimizing an agent’s instructions follows the same process, just applied to behavior across multiple steps rather than a single response:

Baseline agent instructions -> evaluate against realistic multi-step
scenarios (Module 20, applied to agent behavior as covered in Module
19) -> diagnose specific failures (agent skipped a required check?
looped unnecessarily?) -> revise the specific rule that caused the
failure -> re-evaluate the FULL scenario set, not just the one
previously-failing case, to confirm the fix didn't break something
else -> version and deploy.

7. How Is This Used in AI?

🤖 How Is This Used in AI?

Serious AI product teams treat prompt optimization as an ongoing engineering discipline — not a one-time task, but a repeatable process applied whenever a prompt is created, whenever the underlying model changes (Module 26), or whenever real-world usage reveals new failure patterns not covered by the original test set.

8. When Should You Run a Full Optimization Pass?

Before deploying any production-bound prompt for the first time
After a model change (Module 26) — re-evaluate, don’t assume the existing prompt is still optimal
When real-world usage reveals new failure patterns your original test set didn’t cover

9. When Is a Lighter Touch Enough?

Quick, low-stakes, personal prompts don’t need the full process — Module 14’s lighter iteration loop is usually sufficient

10. Common Mistakes

Incorrect idea

Optimizing for efficiency before correctness is established.

Why it is incorrect

As emphasized directly, this wastes effort on a prompt that doesn’t actually work reliably yet.

Incorrect idea

Treating a single evaluation pass as final.

Why it is incorrect

Real-world usage often reveals new failure patterns over time — optimization is an ongoing practice (connecting directly to Module 21’s versioning), not a box to check once.

Incorrect idea

Chasing accuracy improvements without considering cost, or vice versa.

Why it is incorrect

As shown directly, the right balance is a genuine, deliberate trade-off decision, not an automatic “more accurate is always better” or “cheaper is always better” rule.

11. Limitations

This process requires real, ongoing time and effort — really worth it for production systems, potentially excessive for casual, low-stakes use
Optimization can only improve a prompt within what prompting itself can achieve — some tasks may have inherent limits regardless of how well the prompt is optimized (Module 22’s honest limitations on hallucination, for instance)
There’s no single “finished” state — usage patterns, models, and requirements can all change over time, requiring the process to be revisited periodically

Analogy: Building a Race Car Think of systematic prompt optimization in terms of engineering a race car:

Step 1: The Engine (Correctness First): You build the engine. Does the car start and drive forward when you push the gas? (Does the prompt generate the right answer at all?).

Step 2: The Suspension & Tires (Reliability Second): You take the car out to rain, mud, curves, and hills. Does it keep driving safely under diverse weather conditions? (Does it hold up across your test dataset?).

Step 3: Aerodynamics & Weight Reduction (Efficiency Last): Once the engine is bulletproof and the handling is safe, you strip away heavy steel frame pieces, replace glass windows with polycarbonate sheets, and adjust tire pressure. (Trim prompt word padding, set temperature dials).

If you strip weight (Efficiency) before the engine even turns on (Correctness), you just have a light box of metal that goes nowhere.

📊 Visual Flowchart: The Prompt Optimization Hierarchy

Here is the strict order of optimization phases to avoid wasted effort:

graph TD
    classDef first fill:#e74c3c,stroke:#333,stroke-width:1px,color:#fff;
    classDef second fill:#f1c40f,stroke:#333,stroke-width:1px,color:#fff;
    classDef third fill:#2ecc71,stroke:#333,stroke-width:1px,color:#fff;

    Sub1["1. Correctness Phase (Engine)"]:::first --> Q1{"Does output match task?"}:::first

    Q1 -->|Yes| Sub2["2. Reliability Phase (Suspension)"]:::second
    Q1 -->|No| Fix1["Fix instructions / schema (Anatomy)"]:::first

    Sub2 --> Q2{"Does it pass all 50 test cases?"}:::second
    Q2 -->|Yes| Sub3["3. Efficiency Phase (Aerodynamics)"]:::third
    Q2 -->|No| Fix2["Add edge-case rules / examples"]:::second

    Sub3 --> Q3["Trim words / set Temperature = 0"]:::third

12. Quick Reference — The Whole Idea in One Diagram

Correctness FIRST (Module 2, 8, 9)
   ↓
Reliability SECOND (Module 14, 20 -- test across varied inputs)
   ↓
Efficiency LAST (Module 25 -- trim length, Module 26 -- tune parameters)
   ↓
Version and deploy (Module 21)
   ↓
Revisit when: model changes, new failure patterns emerge, usage grows

13. Prompts in Code — Calling an LLM

Here’s how a full optimization pass actually looks in code — combining evaluation, comparison across versions, and a cost/accuracy trade-off decision in one place.

Example 1 — Simple

Running the baseline prompt against a small test set to establish a starting accuracy number.

import anthropic

client = anthropic.Anthropic()

test_cases = [
    {"input": "order #4471 never showed up", "expected": "4471"},
    {"input": "no order number here, just a question", "expected": "NOT_FOUND"},
]

def extract_v1(message: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-4-6", max_tokens=20,
        messages=[{"role": "user", "content":
                   f"Extract the order number from this message: {message}"}]
    )
    return response.content[0].text.strip()

correct = sum(extract_v1(c["input"]) == c["expected"] for c in test_cases)
print(f"v1 accuracy: {correct}/{len(test_cases)}")

Example 2 — Intermediate

Comparing a baseline and a revised version’s accuracy AND token usage side by side — correctness and cost, evaluated together.

import anthropic

client = anthropic.Anthropic()

test_cases = [
    {"input": "order #4471 never showed up", "expected": "4471"},
    {"input": "no order number here, just a question", "expected": "NOT_FOUND"},
    {"input": "Order: 8823", "expected": "8823"},
]

def evaluate_version(prompt_template: str, test_cases: list) -> dict:
    correct = 0
    total_tokens = 0
    for case in test_cases:
        response = client.messages.create(
            model="claude-sonnet-4-6", max_tokens=20, temperature=0,
            messages=[{"role": "user", "content": prompt_template.format(msg=case["input"])}]
        )
        actual = response.content[0].text.strip()
        correct += (actual == case["expected"])
        total_tokens += response.usage.input_tokens + response.usage.output_tokens
    return {"accuracy": correct / len(test_cases), "avg_tokens": total_tokens / len(test_cases)}

v1 = "Extract the order number from this message: {msg}"
v2 = "Extract the order number from this message. Return only digits, " \\
     "no prefix. If none is mentioned, return NOT_FOUND: {msg}"

for label, template in [("v1", v1), ("v2", v2)]:
    result = evaluate_version(template, test_cases)
    print(f"{label}: accuracy={result['accuracy']:.0%}, avg_tokens={result['avg_tokens']:.0f}")

Example 3 — Production Grade

A full optimization report generator that runs multiple candidate versions against a test set, ranks them by a combined accuracy/cost score, and recommends which version to deploy — turning the manual “correctness first, then efficiency” process into a repeatable, automatable tool.

import anthropic

client = anthropic.Anthropic()

TEST_CASES = [
    {"input": "order #4471 never showed up", "expected": "4471"},
    {"input": "no order number here, just a question", "expected": "NOT_FOUND"},
    {"input": "Order: 8823", "expected": "8823"},
    {"input": "order# 4471", "expected": "4471"},
]

CANDIDATE_VERSIONS = {
    "v1": "Extract the order number from this message: {msg}",
    "v2": "Extract the order number. Return only digits, no prefix. "
          "If none is mentioned, return NOT_FOUND: {msg}",
    "v3": "Extract the order number, stripping any '#' or 'Order:' "
          "prefix. Return only digits, no prefix or symbols. If no "
          "order number is mentioned, return exactly NOT_FOUND: {msg}",
}

def evaluate_version(template: str) -> dict:
    correct = 0
    total_tokens = 0
    for case in TEST_CASES:
        response = client.messages.create(
            model="claude-sonnet-4-6", max_tokens=20, temperature=0,
            messages=[{"role": "user", "content": template.format(msg=case["input"])}]
        )
        actual = response.content[0].text.strip()
        correct += (actual == case["expected"])
        total_tokens += response.usage.input_tokens + response.usage.output_tokens
    return {"accuracy": correct / len(TEST_CASES), "avg_tokens": total_tokens / len(TEST_CASES)}

print(f"{'Version':<6} {'Accuracy':<10} {'Avg Tokens':<12}")
best_version, best_score = None, -1
for label, template in CANDIDATE_VERSIONS.items():
    result = evaluate_version(template)
    # Simple combined score: prioritize accuracy heavily, penalize token cost lightly
    score = result["accuracy"] - (result["avg_tokens"] / 10000)
    print(f"{label:<6} {result['accuracy']:.0%}       {result['avg_tokens']:.0f}")
    if score > best_score:
        best_version, best_score = label, score

print(f"\\nRecommended version: {best_version}")

This is exactly the correctness-first, efficiency-second philosophy from Section 3, made concrete: accuracy dominates the combined score, with token cost as a real but secondary consideration — a deliberate, reviewable trade-off rather than an ad hoc choice.

When to use it—and when not to

Use it when:

a prompt has measurable recurring failures.
production cost or response time needs improvement.

Do not rely on it when:

there is no representative test collection.
changes are judged from one attractive output.

14. Interview Questions

Q: Why should prompt optimization address correctness and reliability before efficiency, rather than optimizing all three at once?

Ans: Optimizing for efficiency (shorter prompts, cheaper models) before correctness and reliability are established risks either trimming content that was actually necessary, or investing effort optimizing a prompt that doesn’t reliably work yet in the first place. Establishing that a prompt is correct and reliable across varied test cases first provides a solid, verified foundation — efficiency improvements can then be measured against that baseline to confirm they don’t reintroduce reliability problems.

Q: How would you decide whether a small accuracy loss is an acceptable trade-off for a significant token cost reduction?

Ans: This depends on the real-world cost of the remaining errors in the specific context — for a high-volume, low-stakes task, a small accuracy loss might be entirely acceptable given substantial cost savings at scale. For a high-stakes task where errors have real consequences, even a small accuracy reduction might not be worth any cost savings. This is a genuine business and risk decision, not something a purely technical “shorter is always better” or “more accurate is always better” rule can answer on its own.

Q: What role does versioning (Module 21) play in a systematic prompt optimization process?

Ans: Versioning provides the structure for comparing candidate improvements against the current production version in a tracked, reviewable way — each optimization pass produces a new candidate version, evaluated against the same test set as the current version, with the decision to deploy (or not) based on that direct comparison. This makes the optimization process auditable and reversible, rather than making untracked changes and hoping they were improvements.

Q: Why might a prompt optimization process need to be revisited periodically, rather than being a one-time task?

Ans: Real-world usage patterns can reveal new failure cases the original test set didn’t anticipate, the underlying model might be updated or changed (Module 26), and business requirements or acceptable cost/accuracy trade-offs can shift over time. A prompt that was fully optimized under one set of conditions isn’t guaranteed to remain optimal indefinitely — ongoing evaluation and periodic re-optimization is part of maintaining a really reliable production system.

15. What You Should Remember

Prompt optimization is a synthesis of tools already covered: iteration, evaluation, versioning, token economics, and model/ parameter choices — not a new, separate skill.
The right order is correctness first, reliability second, efficiency last — optimizing efficiency before the other two are established wastes effort.
Accuracy vs. cost is a genuine trade-off decision, not an automatic rule — the right balance depends on the specific stakes and scale of the task.

16. Quick Practice

Take a prompt you’ve iterated on earlier in this course. Walk through the full optimization process from Section 2 for it: what would your test cases be, what specific failure would you look for, and how would you decide whether a shorter version is an acceptable trade-off?

17. Next Step

Next: Module 28 — Production Prompt Engineering — bringing everything from Level 6 together into a complete picture of how a prompt fits into a real, production AI application, including observability, fallback strategies, and human review.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026