Intermediate level · Module 1: Feature Engineering and Pipelines · Lesson 3 of 3
Pipelines
Chain preprocessing and the model into one object that is trained, evaluated, tuned and saved together.
In this lesson you will
- Build a Pipeline with preprocessing and a model
- Explain how pipelines prevent leakage in cross-validation
Key concepts
Pipelinemake_pipelineleakage in cross-validationone object to deploy
The lesson
A Pipeline chains steps: Pipeline([("prep", column_transformer), ("model", LogisticRegression())]). Calling fit runs fit_transform on each step and fits the model; predict transforms new data and predicts.
When cross-validation is run on a pipeline, preprocessing is refitted inside each fold on that fold's training part only. Preprocessing outside the pipeline would leak information from validation folds.
A pipeline is also what you save and deploy: new data goes through exactly the same steps as training data, removing a common source of production bugs.
Step by step
- Combine the ColumnTransformer and a classifier in a Pipeline.
- Fit and evaluate it.
- Swap the classifier without changing preprocessing.
- Show the leakage difference between scaling outside and inside the pipeline.
Practice activity
Build the full pipeline for the agent liquidity model.
Expected result: One pipeline object that prepares data and predicts, used for all later evaluation.
Check your understanding
-
Why do pipelines prevent leakage during cross-validation?
Show answer
Preprocessing is fitted only on each fold's training part.
-
What is the deployment benefit of a pipeline?
Show answer
New data is processed exactly like training data.
Taught live by an instructor — online or in person. Free course lessons are for self-study and do not lead to a certificate — see registration options.