Deep Learning (DL) · Beginner level

Beginner level · Module 3 of 5 · 10 guided hours

Sequence Models for Text and Time

Represent sequences such as text and sensor readings, and understand recurrent networks and the move to attention.

Learning objectives

  • Represent words and sequences as numbers with embeddings
  • Explain how recurrent networks and LSTMs process sequences
  • Describe where sequence models are used and why attention replaced many of them

Topics

  • sequence data
  • tokenisation and vocabularies
  • embeddings
  • recurrent networks and LSTMs
  • sequence classification
  • from RNNs to attention

Lessons

  1. Lesson 1 Representing Sequences and Words

    Turn text into sequences of numbers and learn why embeddings capture meaning better than one-hot codes.

    tokenvocabularyone-hot vectorembedding

  2. Lesson 2 Recurrent Networks and LSTMs

    Process sequences step by step with recurrent networks, and see how LSTMs keep longer memory.

    hidden staterecurrent networkvanishing gradientsLSTM

  3. Lesson 3 Where Sequence Models Are Used

    Survey real uses of sequence models and understand why attention-based transformers now dominate many tasks.

    speech and translationtime-series sensorsattentiontransformers

Practical activity

Train a small LSTM to classify short SMS messages as appointment reminders, payment notices or other, using a provided synthetic data set.

Expected outcomes

  • Understanding of how networks process sequences
  • A working sequence classifier

Assessment

Short quiz on embeddings and recurrence, plus the trained classifier with an error analysis.

Free taster lesson Recurrent Neural Networks (RNNs) for Sequence Data Read a free self-study lesson on this topic in the Free Course Library.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.