Advanced level · Module 1: Ensemble Methods · Lesson 3 of 4
Stacking and Model Combination
Combine different model types with stacking and decide when the extra complexity is worth it.
In this lesson you will
- Build a stacking ensemble
- Judge whether the gain justifies the complexity
Key concepts
stackingmeta-modelout-of-fold predictionscomplexity cost
The lesson
Stacking trains a meta-model on the predictions of several different base models — for example logistic regression, a random forest and gradient boosting. It can beat each individual model when they make different mistakes.
The meta-model must be trained on out-of-fold predictions, which StackingClassifier handles with internal cross-validation; otherwise it learns from predictions on data the base models had already seen.
Stacked models are slower and harder to maintain and explain. Use them only when the improvement matters for the decision.
Step by step
- Build a StackingClassifier with three base models.
- Compare it with the best single model.
- Measure training and prediction time.
- Decide whether to keep it.
Practice activity
Evaluate a stacking ensemble for the food-insecurity task and write a keep-or-drop decision.
Expected result: A comparison including accuracy and costs, with a reasoned decision.
Check your understanding
-
What does the meta-model in stacking learn from?
Show answer
Out-of-fold predictions of the base models.
-
When is stacking not worth it?
Show answer
When the small gain does not justify extra complexity, time and maintenance.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.