Evaluation

The last two articles covered specific tools — Baseline and Benchmark — for judging a model’s performance. This article names the broader activity both of them belong to: evaluation.

The simple definition

Evaluation is the overall process of measuring how well a model performs at its intended task. It’s the umbrella practice; baselines and benchmarks are two specific tools used within it, and the metrics covered in the rest of this phase — Accuracy, Precision, Recall, and F1 Score — are the specific numbers evaluation actually produces. If Validation, from the previous phase, was about the broad discipline of checking whether a model genuinely works, evaluation is the concrete, technical activity of measuring that — the actual act of running a model against data and quantifying how it did.

Why evaluation needs to be more careful than it sounds

It’s tempting to think evaluation just means “check if the model was right.” In practice, being right is rarely a single, simple yes-or-no fact, and how you measure “right” can dramatically change the picture a model’s performance paints. Recall from the Baseline article that a bare accuracy number can be dangerously misleading on an imbalanced dataset. That’s not a one-off quirk — it’s a genuine, recurring reason evaluation needs multiple, carefully chosen metrics rather than one single number, a theme the rest of this phase develops in full.

flowchart LR
    A[Trained Model] --> B[Run on held-out data]
    B --> C[Compare predictions to Ground Truth]
    C --> D[Calculate specific metrics: Accuracy, Precision, Recall, F1]
    D --> E[Interpret what those metrics actually mean for this task]

What evaluation actually requires

Evaluation depends directly on infrastructure this glossary has already covered in earlier phases. It needs genuinely held-out data — Test Data or Validation Data, never touched during training — because evaluating a model on data it already trained on tells you almost nothing about real performance, as the Overfitting article explained at length. It needs reliable Ground Truth to compare predictions against, since an evaluation is only as trustworthy as the correct answers it’s measured against. And it needs the right metric for the task at hand — which is exactly the gap the remaining articles in this phase fill in.

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of a doctor evaluating a patient’s health. It’s not one single measurement — blood pressure, cholesterol, heart rate, and several other specific readings each capture a different dimension of overall health, and a doctor looks at the whole picture rather than trusting any single number alone to tell the complete story.

Where this breaks down: A doctor applies rich clinical judgment to weigh conflicting signals against each other. Model evaluation is far more mechanical — each metric is calculated by a fixed, precise formula comparing predictions to ground truth, with the judgment mostly happening beforehand, in choosing which metrics matter for a given task, rather than in interpreting the numbers themselves once they’re calculated.

Why one number is rarely the whole story

This is worth stating directly, since it’s the central lesson the rest of this phase builds on. A single evaluation number — even a well-chosen one — can hide important, task-specific details. A medical diagnostic model and a spam filter can both report “95% accuracy,” but the real-world consequences of that remaining 5% of errors are wildly different: a missed cancer diagnosis is a far more serious mistake than a missed spam email, even though both might contribute equally to the same overall accuracy score. This exact tension — that not all errors are equally costly — is precisely why the next few articles introduce metrics like Precision and Recall, which distinguish between different types of mistakes rather than lumping them all into one undifferentiated number.

Evaluation begins before running the model

flowchart TD
    A[Define the real-world goal] --> B[Name costly mistakes]
    B --> C[Choose representative test data]
    C --> D[Choose suitable metrics and thresholds]
    D --> E[Run the model]
    E --> F[Inspect totals, slices and examples]
    F --> G[Compare with baseline and requirements]
    G --> H[Decide: improve, deploy or reject]

For a medical screening model, “get high accuracy” is not a complete goal. A useful evaluation plan might say:

• Detect at least 95 of every 100 patients who have the condition.
• Keep unnecessary follow-up tests below an agreed level.
• Check performance separately across age groups and scanning devices.
• Measure prediction speed and failure behavior when an image is blurry.
• Compare results with the current clinical process, which is the baseline.

Offline evaluation and production evaluation

Offline evaluation uses prepared, labelled data before deployment. It is repeatable and safer, but it cannot reproduce every situation in the changing real world.

Production evaluation watches the deployed system using monitoring, delayed ground truth, user feedback, controlled experiments, and human review. It can reveal drift and unexpected behavior, but must be designed carefully so users are not exposed to avoidable harm.

A concrete example, layered

For a simple beginner example: evaluating a model that predicts whether a coin flip will land heads means comparing its predictions against the actual outcomes across many flips, and calculating what fraction it got right — a simple case where a single accuracy number genuinely does tell the whole story, since both types of mistakes (predicting heads when it was tails, or vice versa) are equally costly. For a production example: evaluating a large language model involves running it against a whole suite of different benchmarks and metrics simultaneously — general knowledge (MMLU), coding ability, mathematical reasoning, and increasingly, human preference ratings on open-ended conversation, as covered in the Ground Truth and Labeling articles — precisely because no single number can capture something as broad and multidimensional as “how good is this chatbot,” a genuinely more complex evaluation problem than checking whether a fraud model correctly flagged one transaction.

Real AI-agent example: evaluation improved a safety monitor

OpenAI’s Operator System Card describes a monitor for detecting prompt-injection attempts displayed on webpages. On 77 red-team attacks, it reached 99% recall and 90% precision, missing one borderline attempt. It also flagged 46 of 13,704 benign screens from normal use.

After new attacks were discovered during red teaming, the reported recall improved from 79% to 99% in one day. Evaluation became an engineering feedback loop:

flowchart LR
    A[Discover new attacks] --> B[Measure missed attacks]
    B --> C[Update monitor]
    C --> D[Re-evaluate]
    D --> E[Recall rises from 79% to 99%]

Evaluation did more than produce a score. It exposed failures, guided an update, and checked whether the protection improved.

Common misconception

A frequent assumption: that evaluation is a simple, single step happening once, right after training finishes. In practice, as this article and the previous Validation article both established, evaluation happens repeatedly and for different purposes throughout a model’s life — during development, using validation data to guide tuning decisions; once, honestly, using held-out test data; against external benchmarks, for comparison with other models; and continuously after deployment, monitoring real-world performance for the kind of drift discussed in the Validation article. “Evaluation” describes an ongoing discipline, not a single checkbox.

Where this fits in what comes next

You now understand the broad activity of evaluation, and why a single number rarely captures everything that matters. The next article, Accuracy, covers the single most familiar evaluation metric — and, as this article has already hinted, exactly where its simplicity becomes a genuine limitation.

In one sentence

Evaluation is the overall, ongoing process of measuring how well a model actually performs, and its central lesson — that a single number rarely tells the whole story — is precisely why the rest of this phase introduces several distinct, complementary metrics rather than settling for just one.

Related Terms

• Baseline
• Benchmark
• Validation
• Test Data
• Accuracy