Precision, Recall, and F1: Which Mistakes Matter?
Accuracy is not enough when errors have different costs
A fraud detector, a search result, and a medical screen do not value the same mistakes. Precision and recall focus on the positive class from two different directions.
Key Points
- Precision starts with predicted positives.
- Recall starts with actual positives.
- F1 combines precision and recall into one harmonic mean.
- The positive class must be defined before these metrics can be interpreted.
Precision asks whether positive alerts deserve trust
Precision = TP ÷ (TP + FP). Of all cases the model called positive, how many were truly positive? High precision means fewer false alarms among positive predictions.
A spam filter with high precision sends very few legitimate messages to spam, but it may still miss many spam messages if it is cautious.
Key Points
- The denominator is all predicted positives.
- Precision is undefined when there are no predicted positives, so libraries need a convention.
- Precision matters when investigating a positive alert is expensive or disruptive.
- Precision alone says nothing about positives the model missed.
Recall asks how many real positives were found
Recall = TP ÷ (TP + FN). Of all actual positive cases, how many did the model catch? High recall means fewer missed positives.
Key Points
- The denominator is all actual positives.
- Recall is also called sensitivity or true positive rate.
- Recall matters when missing a positive is dangerous or costly.
- Recall alone says nothing about how many false alarms were raised.
Calculate precision and recall
1tp, fp, fn = 40, 10, 20
2
3precision = tp / (tp + fp)
4recall = tp / (tp + fn)
5
6print(f"Precision: {precision:.3f}")
7print(f"Recall: {recall:.3f}")Precision: 0.800 Recall: 0.667
The model’s positive alerts are correct 80% of the time, but it finds about 66.7% of the actual positives. The interpretation depends on what positive means in the application.
F1 rewards balance
F1 is the harmonic mean of precision and recall: F1 = 2 × precision × recall ÷ (precision + recall). The harmonic mean is pulled down when either value is low, so F1 rewards a balance rather than allowing one metric to hide the other.
1precision = 0.8
2recall = 2 / 3
3f1 = 2 * precision * recall / (precision + recall)
4print(f"F1: {f1:.3f}")F1: 0.727
Key Points
- F1 is between 0 and 1.
- F1 is high only when both precision and recall are reasonably high.
- F1 does not use true negatives directly.
- F1 is not automatically the right summary when precision and recall have very different business costs.
The threshold creates a tradeoff
When the positive threshold falls, more cases are called positive. Recall often rises because fewer positives are missed, while precision can fall because more negative cases become false alarms. The relationship is a pattern to measure, not an unbreakable law for every finite dataset.
Key Points
- Choose a threshold with validation data.
- Plot or tabulate precision and recall at several thresholds.
- Use a cost or service constraint when one kind of error has a hard limit.
- Keep probabilities if downstream users need different thresholds.
Threshold low
Flags almost everything
Catches nearly every real case and buys a pile of false alarms with it. High recall, poor precision. Somebody has to work through that pile.
Threshold high
Flags only the obvious
Almost everything it flags is real, and it walks past the rest. High precision, poor recall. The misses are invisible until they are expensive.
Choose the metric from the action
Key Points
- High recall: disease screening, safety alerts, finding risky transactions for review.
- High precision: expensive manual investigations, customer contact, strict automated actions.
- F1: a single balanced summary when both positive precision and recall matter.
- Neither alone: when costs are asymmetric, report the confusion counts and the operational cost.
“Optimise recall” is incomplete unless you also state the acceptable false-positive burden. Metrics are a translation of the decision, not a contest for the largest decimal.
Macro, micro, and weighted summaries
For multiclass problems, calculate metrics per class and combine them. Macro averaging gives each class equal weight. Weighted averaging weights classes by support. Micro averaging pools the decisions before calculating the metric.
Key Points
- Macro can expose poor performance on a small class.
- Weighted scores reflect the observed class mix.
- Micro is influenced strongly by common classes.
- Report per-class results when a rare class matters.
Where people get confused
Key Points
- “Precision and recall are the same.” Precision conditions on predicted positives; recall conditions on actual positives.
- “High recall means few false positives.” Recall ignores false positives; precision describes them among alerts.
- “F1 is always best.” F1 is useful for balance but does not encode every operational cost.
- “Accuracy and F1 must move together.” A model can improve positive-class recall while overall accuracy falls.
- “A threshold is part of the learned weights.” The threshold is a separate decision setting.
Quick check
What does precision measure?
Among the cases predicted positive, precision is the fraction that are actually positive. It is TP divided by TP plus FP.
What does recall measure?
Among all actual positive cases, recall is the fraction the model finds. It is TP divided by TP plus FN.
Why does F1 use a harmonic mean?
It keeps a low precision or low recall from being hidden by a high value of the other metric, rewarding a reasonably balanced result.
When might recall matter more than precision?
When missing a positive case is especially harmful, such as an initial safety or disease screen. The acceptable false-positive burden still needs to be specified.
Recap
Key Points
- Precision measures trust in positive alerts.
- Recall measures coverage of actual positives.
- F1 summarises their balance.
- Thresholds, class definitions, and error costs control the interpretation.
- The next lesson explains why class imbalance can make familiar metrics misleading.
Theia
Can you explain this concept?
Practise Precision, Recall, and F1: Which Mistakes Matter? with realistic interview questions. Choose spoken or typed answers, get feedback on your reasoning, then return to the material that needs work.
Sign in to practise this conceptThe session stays focused on this lesson. Your selected interview level sets the bar.