Advanced level · Module 3: Open Models and Customisation · Lesson 1 of 4
Running Open Models Locally
Download and run open-weight language models on local hardware.
In this lesson you will
- Run a quantised open model locally
- Check licences and hardware needs
Key concepts
open-weight modelslicences and acceptable usequantised model fileslocal inference
The lesson
Open-weight models publish their trained weights, so they can run on your own computers. Licences differ: some allow commercial use, some restrict it, and many have acceptable-use rules.
Quantised versions reduce memory needs so smaller models run on ordinary laptops, more slowly than on servers but with data never leaving the machine.
Local models are useful for privacy-sensitive data and offline work. They are usually less capable than the largest hosted models, so evaluate them on your own tasks.
Step by step
- Read the licence of a small open model.
- Run a quantised version locally.
- Measure speed and memory.
- Ask it five capstone questions.
Practice activity
Run a local model and record its speed, memory use and answer quality.
Expected result: Measurements and sample answers from a locally running model.
Check your understanding
-
What does quantisation allow?
Show answer
Running models with less memory, for example on laptops.
-
Why read an open model's licence?
Show answer
Licences restrict how models may be used.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.