Intermediate level · Module 2: Working with AI Model APIs · Lesson 3 of 3
Cost, Latency and Provider Choice
Estimate and control the cost and speed of AI applications, and choose providers responsibly.
In this lesson you will
- Estimate cost from token counts
- Compare providers on quality, cost, speed and data policies
Key concepts
token pricinglatencycaching and batchingdata policies and location
The lesson
Costs scale with tokens: long prompts, long documents and long answers cost more. Estimate cost per task and per month before launching.
Reduce cost and latency by using smaller models where they are good enough, shortening prompts, caching repeated answers and batching work that is not urgent.
Compare providers on quality for your task, price, speed, reliability, data retention policies and where data is processed. Avoid building on a single provider without a fallback.
Step by step
- Calculate the cost of processing 1,000 messages.
- Compare a small and a large model on 20 messages.
- Add caching for repeated inputs.
- Compare two providers' data policies.
Practice activity
Write a cost and provider recommendation for the feedback-processing script.
Expected result: A recommendation with cost estimates, quality comparison and data policy review.
Check your understanding
-
Name two ways to reduce AI API costs.
Show answer
For example using smaller models and shortening prompts (or caching).
-
Why compare providers' data policies?
Show answer
They differ in how they store and use submitted data.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.