Data Drift Detection Fundamentals
Catch Production Data Drifting Away From Your Training Data With KS, PSI, and Chi-Squared Tests Before Accuracy Quietly Drops.
By Sandip Das · · 5 min read
💡
Hey — It's Sandip Das 👋
In Day 39 we put a Great Expectations gate in front of the pipeline so broken batches never reach training. But data can pass every rule and still be wrong for your model: valid values, different world.
Today we learn to detect drift: compare production data against training data with statistical tests, and let Evidently AI tell us which features moved.
In Day 39 we put a Great Expectations gate in front of the pipeline so broken batches never reach training. But data can pass every rule and still be wrong for your model: valid values, different world.
Today we learn to detect drift: compare production data against training data with statistical tests, and let Evidently AI tell us which features moved.
📚 Key Learnings
- What data drift is and why valid data can still break a model
- Covariate shift, label (prior) drift and concept drift, with examples
- The three workhorse tests: KS, PSI and chi-squared, and when to use each
- Running an Evidently drift report on reference vs production iris data
- Hands-on: simulate drift, find the drifted features, fail on drift in a script
🧠 Learn here
Read the full article on learnxops.com.
Topics: 100 Days of MLOps, MLOps, ML, Monitoring