Machine Learning (ML) · Beginner level

Beginner level · Module 4 of 5 · 11 guided hours

Evaluating Machine Learning Models

Measure model performance with the right metrics for the decision, and detect and reduce overfitting.

Learning objectives

  • Evaluate regression models with MAE and RMSE
  • Evaluate classifiers with a confusion matrix, precision, recall and F1
  • Recognise overfitting and use validation to choose model complexity

Topics

  • MAE, RMSE and R²
  • confusion matrix
  • accuracy, precision, recall, F1
  • class imbalance and accuracy
  • overfitting and underfitting
  • validation sets

Lessons

  1. Lesson 1 Measuring Regression Errors

    Measure how far numeric predictions are from the truth with MAE, RMSE and R².

    mean absolute errorroot mean squared errorR²error in real units

  2. Lesson 2 The Confusion Matrix and Classification Metrics

    Count each kind of right and wrong prediction and choose metrics that match the cost of mistakes.

    true/false positives and negativesprecisionrecallF1 and thresholds

  3. Lesson 3 Overfitting, Underfitting and Validation

    Diagnose models that memorise or oversimplify, and use a validation set to choose model complexity.

    overfittingunderfittingvalidation setbias–variance trade-off

Practical activity

Evaluate the dropout classifiers with a confusion matrix and choose a threshold based on the cost of missing an at-risk student versus a false alarm.

Expected outcomes

  • Metrics chosen for the real decision
  • Ability to spot overfitting

Assessment

Written and practical evaluation of two provided models; marked on metric calculation, interpretation and recommendation.

Free taster lesson Model Evaluation & Avoiding Overfitting Read a free self-study lesson on this topic in the Free Course Library.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.