Beginner level · Module 4: Evaluating Machine Learning Models · Lesson 2 of 3
The Confusion Matrix and Classification Metrics
Count each kind of right and wrong prediction and choose metrics that match the cost of mistakes.
In this lesson you will
- Build and read a confusion matrix
- Calculate accuracy, precision, recall and F1 and choose the right one
Key concepts
true/false positives and negativesprecisionrecallF1 and thresholds
The lesson
A confusion matrix counts true positives (at-risk students correctly flagged), false positives (false alarms), false negatives (at-risk students missed) and true negatives.
Precision is the share of flagged cases that were truly positive: of the students flagged, how many were really at risk? Recall is the share of real positives that were found: of the students at risk, how many did we flag? F1 combines both.
Accuracy misleads when classes are imbalanced: if 5% of students drop out, a model that predicts 'no' for everyone is 95% accurate and completely useless. Choose metrics — and the threshold — by asking which mistake is worse for the people affected.
Step by step
- Build a confusion matrix for the dropout model.
- Calculate precision, recall and F1 by hand and check with scikit-learn.
- Change the threshold and watch precision and recall trade off.
- Choose a threshold and justify it.
Reading a confusion matrix for a malaria screening model
Accuracy, precision, recall and F1 calculated step by step from 200 test cases.
| Item | Working | Amount |
|---|---|---|
| True positives (malaria, flagged) | counted | 45 |
| False negatives (malaria, missed) | counted | 5 |
| False positives (no malaria, flagged) | counted | 15 |
| True negatives (no malaria, not flagged) | counted | 135 |
| Accuracy | (45 + 135) ÷ 200 | 90% |
| Precision | 45 ÷ (45 + 15) | 75% |
| Recall (sensitivity) | 45 ÷ (45 + 5) | 90% |
| Specificity | 135 ÷ (135 + 15) | 90% |
| F1 score | 2 × 0.75 × 0.90 ÷ (0.75 + 0.90) | 0.82 |
Explanation
Recall answers the clinic's most important question: of the 50 patients who had malaria, the model flagged 45 and missed 5. Precision says that of the 60 patients flagged, 45 really had malaria — 1 in 4 flags was a false alarm.
For screening, missing a sick patient is usually worse than an extra test, so the clinic might lower the threshold to raise recall, accepting more false alarms. That is a decision for health staff, informed by these numbers.
Check your understanding
The four counts add up to 200 test patients, and true positives plus false negatives equal the 50 patients who had malaria.
Practice activity
Complete the threshold choice from the practical activity and justify it in terms of missed students and false alarms.
Expected result: A threshold choice backed by the confusion matrix and a clear explanation of its trade-off.
Check your understanding
-
What does recall measure?
Show answer
The share of real positive cases that the model found.
-
Why can accuracy be misleading?
Show answer
With imbalanced classes, predicting the majority class gives high accuracy but no useful detection.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.