Machine Learning (ML) · Beginner level

Beginner level · Module 5: Responsible ML and a First Project · Lesson 1 of 3

Data Quality and Bias

Recognise how unrepresentative or flawed data leads to unfair or wrong models, and check performance across groups.

In this lesson you will

  • Identify sources of bias in data and labels
  • Compare model performance across groups

Key concepts

representative samplehistorical biaslabel biasgroup-wise metrics

The lesson

A model learns whatever patterns are in its data, including unfair ones. If data mostly comes from urban schools, the model may work poorly for rural ones; if past decisions were biased, labels based on them carry that bias forward.

Check who is in the data and who is missing. Compare the share of each group in the data with the population the model will be used on.

Evaluate performance separately for important groups, such as girls and boys or each region. A model with good overall recall can still miss most at-risk students in one group.

Step by step

  1. Compare group shares in the data with the target population.
  2. Calculate recall separately for girls and boys.
  3. Calculate recall for rural and urban schools.
  4. Write down any gaps and possible causes.

Practice activity

Run the group-wise check on your dropout model and report the results.

Expected result: Group-wise metrics with an honest discussion of any gaps and their possible causes.

Check your understanding

  1. How can historical data make a model unfair?

    Show answer

    If past decisions were biased, labels based on them teach the model the same bias.

  2. Why calculate metrics per group?

    Show answer

    Good overall results can hide poor results for one group.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.