Artificial Intelligence (AI) · Intermediate level

Intermediate level · Module 3: Prompt Engineering · Lesson 3 of 3

Testing and Versioning Prompts

Treat prompts like code: test them on fixed question sets, version them and avoid regressions.

In this lesson you will

  • Build a prompt test set with expected answers
  • Version prompts and compare results

Key concepts

test setregressionprompt versionsresults log

The lesson

A small change to a prompt can fix one answer and break three others. Keep a fixed test set of questions with expected answers or criteria and run it after every change.

Store prompts in version control with notes on why each change was made, and record each version's results.

Include different question types: simple, multi-part, out-of-scope, ambiguous and adversarial ('ignore your rules and...').

Step by step

  1. Build a 20-question test set with expected answers.
  2. Run versions 1 and 2 on it.
  3. Make a version 3 and run it.
  4. Record results and choose the best version.

Practice activity

Complete the prompt development log from the practical activity.

Expected result: A log of three versions with test results and a justified choice.

Check your understanding

  1. Why run the full test set after every prompt change?

    Show answer

    To catch regressions where a fix breaks other answers.

  2. Name two question types every test set should include.

    Show answer

    For example out-of-scope and adversarial questions.

Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.