MLOps on Kubernetes: Training, Serving, and Monitoring Models
Build a compact, end-to-end MLOps workflow on Kubernetes, from containerized training to model serving and operational monitoring. Practice reproducible training jobs, artifact handling, inference deployment, and basic model and service monitoring through guided labs and a small capstone. The course balances architectural explanations with executable examples within eight hours.
Level: intermediate · 24 lessons · about 8 h · Free · AI-assisted course created by Sandip Das
What you will learn
Package a small model-training workload in a container with explicit dependencies and configuration.
Run and troubleshoot Kubernetes training Jobs using resource requests, logs, and job status.
Preserve model artifacts and training metadata outside ephemeral training containers.
Deploy a trained model behind a Kubernetes Service and validate inference requests.
Monitor inference latency, errors, and basic model-quality or drift signals using a defined baseline.
Demonstrate a repeatable train-to-serve workflow and document validation and rollback steps.
Course outline
1. Workflow Architecture and a Local Kubernetes Environment
Check prerequisites and prepare a cloud-neutral, CPU-only environment for one small scikit-learn training-to-serving workflow.
Verify Prerequisites and Prepare the Cluster(20 min)· free preview Use a short checklist for Python, containers, basic ML, and kubectl; select an existing cluster or a single-node kind cluster and isolate course resources.
Map Training, Artifacts, Serving, and Monitoring(20 min) Draw the end-to-end architecture and distinguish finite training Jobs, replaceable inference Pods, durable artifacts, and service versus model signals.
Run a Local Image and Inspect Its Workload(20 min) Build and load an explicitly tagged test image into kind, or use the existing cluster's image workflow, then inspect status and logs in a guided readiness exercise.
2. Package Reproducible CPU Training
Create a small scikit-learn classification pipeline with explicit configuration, leakage-safe evaluation, and a repeatable container environment.
Define the Model and Evaluation Contract(20 min) Choose a bundled, CPU-friendly classification dataset, split before preprocessing, and establish a simple baseline and task-appropriate quality metric.
Make Training Configuration and Provenance Explicit(20 min) Expose dataset, output path, and seed configuration; record dataset identifiers and exact dependencies while explaining why a seed alone is insufficient.
Build and Test the Training Image(20 min) Write the Dockerfile and dependency lock, exclude unnecessary build content, and verify a local training run produces a model and evaluation metadata.
3. Run and Troubleshoot Kubernetes Training Jobs
Execute bounded training Jobs with external configuration and resource controls, then diagnose common scheduling and container failures.
Declare a Bounded Training Job(20 min) Translate the training command into a batch/v1 Job and use a ConfigMap for non-confidential parameters; keep the local example credential-free.
Set Resources and Verify Job Completion(20 min) Run the Job with CPU and memory requests and limits, inspect its Pods and completion condition, and distinguish scheduling requests from runtime limits.
Diagnose Failed Training Jobs(20 min) Use guided faults and a decision-tree cheat sheet to troubleshoot Pending Pods, image-pull failures, application exits, and OOM termination; finish with a short checkpoint quiz.
4. Preserve Artifacts and Lightweight Run Metadata
Persist trusted model bundles outside training Pods using a local PVC, with run-specific paths and enough provenance to reproduce and promote a model.
Provision Persistent Storage for Model Bundles(20 min) Use a PVC with a verified local StorageClass or prepared local volume; explain single-node lab constraints and why ReadWriteOnce does not mean one Pod.
Publish a Versioned Model and Run Record(20 min) Write a run-specific bundle containing the model, metrics, dataset identifier, seed, configuration, and dependency/image identity; mark publication complete only after validation.
Recover and Validate an Artifact After Pod Deletion(20 min) Delete completed training Pods without deleting their claim, read the bundle from a separate validation Pod, and select a known-good run using a lightweight promotion checklist.
5. Deploy and Validate CPU Inference
Expose the persisted pipeline through FastAPI and Kubernetes, with explicit contracts, model-aware startup, and a stable Service endpoint.
Build a Model-Aware FastAPI Application(20 min) Use the same dependency versions and preprocessing pipeline as training, load a selected trusted bundle during lifespan initialization, and expose prediction and health endpoints.
Deploy the API Behind a Kubernetes Service(20 min) Build the serving image and deploy it with a read-only artifact mount, explicit version selection, CPU and memory settings, labels, and a ClusterIP Service.
Configure Probes and Diagnose Serving Failures(20 min) Protect model initialization with startup probes, gate traffic with readiness, and inject a bad model path or selector fault to practice recovery.
6. Measure Inference Latency and Errors
Add bounded-cardinality Prometheus metrics, deploy a minimal scraper, and inspect user-impacting service signals without installing a full monitoring platform.
Instrument Requests, Errors, and Latency(20 min) Add counters and a seconds-based histogram to the API, using bounded route and response-code labels rather than request IDs or input values.
Collect Metrics with a Minimal Prometheus Setup(20 min) Deploy a small Prometheus instance with a static scrape target, generate test traffic, and verify raw metrics, target health, and collected series separately.
Query Service Signals and Define Actionable Thresholds(20 min) Calculate request rates, error ratios, and histogram-based p95 latency; choose a sustained threshold and diagnostic action, noting sparse-traffic limitations.
7. Monitor Model Quality Against a Baseline
Separate service health from predictive performance and compare labeled quality and unlabeled distributions with explicit reference data.
Define a Quality Baseline and Label Policy(20 min) Reuse the frozen evaluation split and run metadata to define a quality baseline, useful class-level checks, and a plan for delayed production labels.
Compare Reference and Current Data(20 min) Use a small offline comparison script and synthetic shifted samples to inspect feature and prediction distributions, without requiring a separate monitoring platform.
Triage Service, Data, and Quality Signals(20 min) Combine a latency spike, an unlabeled shift, and a labeled quality drop in a guided diagnosis exercise; complete a quiz and interview-style explanations.
8. Capstone: Repeatable Training, Release, and Rollback
Integrate the prepared components into a demonstrable train-to-serve release with validation evidence, a controlled failure, and a verified rollback checklist.
Capstone: Train and Publish a New Candidate(20 min) Reuse the built components to execute a fresh configured Job, persist its complete bundle, and record commands and evidence in a compact release manifest.
Capstone: Release and Validate the Candidate(20 min) Update the Deployment Pod template to pair a selected model bundle with an immutable image identity; run schema, prediction, health, quality, and metric acceptance checks.
Capstone: Verify Rollback and Deliver the Runbook(20 min) Introduce a controlled bad candidate, restore the retained image/model pairing, repeat smoke tests, and deliver a validation-and-rollback checklist plus scoped cleanup instructions.