Advanced level · Module 2: Generative Models · Lesson 3 of 4
Large Language Models
Understand how large language models generate text, why they can be wrong and how they are adapted.
In this lesson you will
- Explain next-token prediction and instruction tuning
- Recognise hallucination and evaluate outputs
Key concepts
next-token predictionpre-training and instruction tuninghallucinationretrieval augmentation
The lesson
Large language models are decoder transformers trained to predict the next token on vast text collections, then tuned to follow instructions and be helpful.
They generate fluent text, but fluency is not accuracy: they can state false facts confidently (hallucination), especially about local topics with little training data.
Retrieval-augmented generation gives the model relevant documents to answer from, which improves accuracy and allows citations. Always evaluate outputs against trusted sources for your use case.
Step by step
- Ask a language model ten questions about local services.
- Check each answer against trusted sources.
- Classify errors.
- Test whether providing a source document improves answers.
Practice activity
Evaluate a language model for a local information task and report its error rate and types.
Expected result: A measured evaluation with error categories and a recommendation.
Check your understanding
-
What is hallucination in language models?
Show answer
Generating confident but false information.
-
How does retrieval augmentation help?
Show answer
It gives the model relevant documents to base answers on.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.