Deep Learning (DL) · Intermediate level

Intermediate level · Module 2: Modern CNN Architectures · Lesson 1 of 3

Deeper Networks and Batch Normalisation

See how stacking more layers improves features and how batch normalisation stabilises training.

In this lesson you will

  • Explain the benefits and difficulties of deeper CNNs
  • Use batch normalisation in a network

Key concepts

network depthreceptive fieldbatch normalisationtraining stability

The lesson

Deeper networks build more complex features from simple ones and see larger parts of the image (a larger receptive field), which improves recognition of whole objects.

Very deep plain networks are hard to train: gradients become unstable and accuracy can get worse as layers are added.

Batch normalisation normalises each layer's outputs within a batch, stabilising training and allowing higher learning rates. It behaves differently in training and evaluation mode — always call model.eval() before evaluating.

Step by step

  1. Train a shallow and a deeper CNN and compare them.
  2. Add batch normalisation to the deeper one.
  3. Compare training stability.
  4. Show the effect of forgetting model.eval().

Practice activity

Report how depth and batch normalisation affect training on the project data.

Expected result: A comparison with loss curves and an explanation of the train/eval mode difference.

Check your understanding

  1. What does batch normalisation do?

    Show answer

    Normalises layer outputs within a batch to stabilise training.

  2. Why call model.eval() before evaluation?

    Show answer

    Layers such as batch normalisation and dropout behave differently during evaluation.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.