F1 Score

The Precision and Recall articles each ended the same way: pointing out that the other metric can be trivially gamed if used alone. A model can reach perfect precision by barely flagging anything, or perfect recall by flagging everything. The F1 score exists specifically to close both of these loopholes at once.

The simple definition

The F1 score is a single number that combines precision and recall into one balanced measure, calculated in a way that punishes a model for being extremely lopsided in either direction. It’s not a simple average — a simple average of a very high precision and a very low recall could still look deceptively decent. The F1 score is calculated using something called the harmonic mean, which specifically penalizes a large gap between the two more heavily than a regular average would.

F1 = 2 × (precision × recall) / (precision + recall)

Why a simple average wouldn’t work here

This is worth understanding concretely, because it’s the entire reason F1 uses this particular formula instead of just averaging. Recall the “flag everything” trap from the Recall article: a model with 100% recall but only 10% precision. A plain average of these two numbers would be 55% — a number that sounds mediocre but reasonable, and completely fails to reflect how genuinely useless that model actually is in practice. The harmonic mean used by F1 handles this very differently: for precision of 10% and recall of 100%, the F1 score comes out to roughly 18% — a number that correctly reflects just how badly unbalanced, and how practically unusable, that model really is. The harmonic mean is specifically sensitive to the smaller of the two values, refusing to let one very high number mask one very low one.

flowchart LR
    A[Precision: 10%] --> C[Simple average: 55%]
    B[Recall: 100%] --> C
    A --> D[F1 Score: ~18%]
    B --> D
    D --> E[F1 correctly reflects the imbalance; the average hides it]

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of grading a two-part exam where a student needs to pass both sections, not just average well across them. A student scoring 100% on the easy section and 10% on the hard section clearly hasn’t mastered the material, even though their simple average (55%) might look passable. A grading system that specifically punishes this kind of lopsided performance — rather than one that lets a strong section compensate for a weak one — better reflects genuine, balanced competence.

Where this breaks down: An exam grader can apply nuanced judgment about why a student did poorly on one section. The F1 score’s harmonic mean is a fixed mathematical formula, applied identically every time, with no awareness of why precision and recall diverge — it simply penalizes the gap between them mechanically, regardless of the underlying cause.

When F1 is the right choice, and when it isn’t

F1 is genuinely useful whenever you want one balanced number and neither precision nor recall is dramatically more important than the other — a fair, general-purpose summary rather than a specialized one. But recall from the Precision and Recall articles that many real tasks do have a genuine priority between the two: cancer screening deliberately favors recall over precision, and content moderation may deliberately favor precision over recall. In those cases, F1’s balanced treatment can actually be the wrong tool — deliberately choosing to weight recall or precision more heavily (using a variant sometimes called F-beta, which lets you tune how much each side counts) is often more appropriate than forcing an even 50-50 balance where the real-world costs genuinely aren’t even.

Calculate F1 from the shared example

The screening model has:

precision = 18 ÷ 26 ≈ 0.692
recall    = 18 ÷ 20 = 0.900

Now combine them using the harmonic mean:

F1 = 2 × (precision × recall) ÷ (precision + recall)
   = 2 × (0.692 × 0.900) ÷ (0.692 + 0.900)
   ≈ 0.783
   ≈ 78.3%

The same formula can be written directly from the confusion-matrix counts:

F1 = 2TP ÷ (2TP + FP + FN)
   = 36 ÷ (36 + 8 + 2)
   = 36 ÷ 46
   ≈ 78.3%

Why F1 punishes an extreme imbalance

Compare two models:

Model	Precision	Recall	Ordinary average	F1 score
A	90%	90%	90%	90%
B	100%	10%	55%	18.2%

Model B is perfectly precise when it makes a positive prediction, but it finds only one tenth of the real positive cases. The ordinary average hides this weakness; F1 falls sharply because one side is very low.

F1 still ignores true negatives. That can be appropriate when the positive class is rare and both false positives and false negatives matter, but it is not automatically the best metric for every task.

A concrete example, layered

For a simple beginner example: a spam filter with 90% precision and 70% recall has an F1 score of roughly 79% — a single number reflecting solid, if imperfect, performance on both fronts, more informative for a general comparison than either number alone. For a production example: research teams comparing different versions of a named-entity-recognition model (a system that identifies people, places, and organizations in text) commonly report F1 as the headline metric specifically because it’s a fair, standard way to summarize performance without the paper needing to separately justify a precision/recall trade-off for every single comparison — it’s become something of a default, general-purpose reporting convention across much of the classification and information-extraction literature for exactly this reason.

Real AI-agent example: combine the published precision and recall

The Operator System Card reports 90% precision and 99% recall for its prompt-injection monitor. Using those rounded published values, we can calculate an approximate F1 score:

F1 = 2 × (0.90 × 0.99) ÷ (0.90 + 0.99)
   = 1.782 ÷ 1.89
   ≈ 0.943
   ≈ 94.3%

The 94.3% is a teaching calculation from the rounded precision and recall, not a separately reported number in the system card. It compresses the two measurements into one value, but the original pair still tells the operational story more clearly: the monitor was tuned to catch nearly all attacks, accepting some false alarms.

Common misconception

A frequent assumption: that F1 is simply “the best” metric, superior to precision, recall, or accuracy, and should always be reported instead of them. This isn’t right — F1 is a useful default for balanced situations, not a universally superior replacement. As the “when F1 is the right choice” section explained, a task with a genuine, deliberate priority between precision and recall is often better served by reporting those two metrics directly, or using a weighted F-beta variant, rather than collapsing them into one number that assumes they matter equally when, for that specific task, they genuinely don’t.

Closing out this phase

This article completes the Evaluation Basics phase, and it’s worth tracing the full arc it built: Baseline gave performance numbers meaning by comparison; Benchmark extended that comparison to a standardized, shared scale across different models; Evaluation named the broader practice both serve; Accuracy introduced the simplest metric and its real danger on imbalanced data; the Confusion Matrix broke predictions into the four specific outcomes that reveal what accuracy hides; Precision and Recall each captured one half of the resulting trade-off; and F1 Score closed the loop by combining both into one balanced, harder-to-game number. From here, the glossary is ready to move from evaluating models into the specific architectures that make today’s most capable systems possible — Deep Learning and Neural Networks.

In one sentence

The F1 score combines precision and recall into a single balanced number that’s genuinely hard to inflate by gaming either metric alone — a solid general-purpose default, though not a universal substitute for reporting precision and recall separately when a task has a real, deliberate priority between the two.

Related Terms

• Precision
• Recall
• Confusion Matrix
• Accuracy
• Evaluation