Deep Learning (DL) · Intermediate level

Intermediate level · Module 1: PyTorch in Depth · Lesson 2 of 3

Datasets and DataLoaders

Load data from folders or files with custom Dataset classes and efficient DataLoaders.

In this lesson you will

  • Write a custom Dataset
  • Configure DataLoader batching, shuffling and workers

Key concepts

Dataset __len__ and __getitem__DataLoadertransforms pipelineshuffling

The lesson

A Dataset tells PyTorch how many samples there are (__len__) and how to load one (__getitem__), for example by reading an image file and its label from a CSV index.

A DataLoader groups samples into batches, shuffles training data each epoch and can load data in parallel with several workers.

Apply transforms in the dataset — resizing, normalising and, for training only, augmentation. Keep validation and test transforms deterministic.

Step by step

  1. Write a Dataset that reads images listed in a CSV.
  2. Create training and validation loaders.
  3. Add training-only augmentation.
  4. Display a batch to check images and labels.

Practice activity

Implement the dataset and loaders for the medicine package project.

Expected result: Loaders that return correctly labelled batches, checked visually.

Check your understanding

  1. Which two methods must a Dataset implement?

    Show answer

    __len__ and __getitem__.

  2. Should validation data be augmented?

    Show answer

    No; validation transforms should be deterministic.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.