Intermediate level · Module 6: Intermediate AI Project · Lesson 2 of 3
Building and Improving
Build the assistant, evaluate it and improve it in measured steps.
In this lesson you will
- Integrate retrieval, prompting and validation
- Improve the system using evaluation results
Key concepts
integrationbaseline evaluationtargeted improvementsregression checks
The lesson
Assemble the pipeline: retrieval, the tested prompt, output checks and logging of tokens and costs.
Run the full evaluation for a baseline, then fix the most important failure categories one at a time — chunking, retrieval settings, prompt rules — re-running the evaluation after each change.
Keep a change log linking each change to its measured effect.
Step by step
- Integrate the full pipeline.
- Run the baseline evaluation.
- Make two targeted improvements.
- Re-run the evaluation after each.
Practice activity
Build and improve the FAQ assistant with a change log.
Expected result: A working assistant with measured improvements over the baseline.
Check your understanding
-
Why change one thing at a time?
Show answer
So each change's effect can be measured.
-
What should the change log record?
Show answer
Each change and its measured effect.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.