Deep Learning (DL) · Beginner level

Beginner level · Module 4: Transfer Learning with Pre-trained Models · Lesson 2 of 3

Feature Extraction

Use a frozen pre-trained network as a feature extractor and train a new classifier head on top.

In this lesson you will

  • Freeze a pre-trained network's layers
  • Replace and train the classifier head

Key concepts

frozen layersclassifier headrequires_gradfast training

The lesson

In feature extraction, keep the pre-trained layers fixed (frozen) and replace the final layer with a new one sized for your classes, such as healthy and three diseases.

Set requires_grad = False for the frozen parameters, so only the new head is trained. Training is fast because most of the network does not change.

This approach is a strong, quick baseline and works well when your images resemble those the model was trained on.

Step by step

  1. Load the pre-trained model.
  2. Freeze all its layers.
  3. Replace the final layer for your classes.
  4. Train the head and evaluate it.
Step-by-step

Transfer learning for a leaf disease classifier

The sequence that turns a model pre-trained on millions of photos into a classifier for local crop leaves.

  1. Choose a pre-trained model

    Pick a small, well-known model (e.g. a ResNet or MobileNet from torchvision), check its licence and size.

  2. Prepare the images

    Resize and normalise exactly as the model expects; add realistic augmentation to training images only.

  3. Replace the head

    Swap the final layer for one with an output per class — healthy plus each disease.

  4. Freeze and train the head

    Set requires_grad = False on pre-trained layers and train only the new head for a quick baseline.

  5. Fine-tune gently

    Unfreeze the last block, use a much smaller learning rate and stop early when validation stops improving.

  6. Evaluate honestly

    Test once on held-out photos from different farms and phones; inspect the mistakes.

Result

A useful classifier from a few hundred labelled photos, trained in minutes rather than days — with its limits known before anyone relies on it.

Practice activity

Train a feature-extraction baseline for the leaf disease classifier.

Expected result: A working baseline with validation accuracy recorded.

Check your understanding

  1. What is trained in feature extraction?

    Show answer

    Only the new classifier head.

  2. How do you freeze a parameter in PyTorch?

    Show answer

    Set its requires_grad attribute to False.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.