Machine Learning (ML) · Advanced level

Advanced level · Module 5 of 6 · 20 guided hours

MLOps and Monitoring

Track experiments, version data and models, monitor production behaviour and govern retraining.

Learning objectives

  • Track experiments and version data and models
  • Monitor input drift, prediction drift and performance
  • Define retraining triggers and governance responsibilities

Topics

  • experiment tracking (e.g. MLflow)
  • data and model versioning
  • model registry
  • drift detection
  • performance monitoring with delayed labels
  • retraining and rollback
  • ownership and governance

Lessons

  1. Lesson 1 Experiment Tracking and Versioning

    Record every training run's data, code, parameters and results, and version models in a registry.

    experiment runparameters and metricsdata versioningmodel registry

  2. Lesson 2 Monitoring Models in Production

    Detect when inputs, predictions or performance change, even before true outcomes are known.

    input driftprediction driftdelayed labelsalert thresholds

  3. Lesson 3 Retraining, Rollback and Governance

    Decide when and how to retrain, roll back safely and assign clear responsibility for models.

    retraining triggersshadow and champion–challengerrollbackmodel ownership

  4. Lesson 4 Production Readiness Review

    Check an ML system against a production readiness checklist before release.

    readiness checklisttesting ML codesecurity and privacyoperational runbook

Practical activity

Track the forecasting experiments, register the chosen model, simulate three months of new data with a price shock and detect and respond to the drift.

Expected outcomes

  • Traceable models and experiments
  • Production monitoring and response plans

Assessment

Practical demonstration of tracking, drift detection and a documented retraining decision.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.