Advanced level · Module 2: AI Agents and Tool Use · Lesson 4 of 4
Evaluating Agents
Evaluate agents on task success, safety and efficiency, not just answer quality.
In this lesson you will
- Design task-based agent evaluations
- Measure success, safety violations and cost
Key concepts
task successtrajectory reviewsafety violationscost and steps per task
The lesson
Evaluate agents on realistic tasks with defined success criteria: did the referral summary contain the correct information and the right template?
Review trajectories — the steps taken — not just the final result. An agent that reaches the right answer by calling forbidden tools has failed.
Track safety violations, steps, time and cost per task, and compare versions on the same task set.
Step by step
- Write ten agent tasks with success criteria.
- Run the agent and record trajectories.
- Score success and safety.
- Report steps and cost per task.
Practice activity
Evaluate the capstone agent on the task set.
Expected result: A task-based evaluation with success, safety and cost results.
Check your understanding
-
Why review an agent's trajectory?
Show answer
A correct result reached through unsafe steps is still a failure.
-
Name two efficiency measures for agents.
Show answer
Steps and cost per task (also time).
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.