How a B2B SaaS Team Cut CI and Batch Costs by 70% Using Spot Instances Without Losing Jobs
A 120-engineer SaaS team moved CI pipelines and nightly batch workloads to Spot/Preemptible instances, slashing compute bills by 70% through checkpointing, retries, and mixed instance-type pools.
Infrastructure · intermediate · 12 min read · 2177 words · By Sandip Das · Published
Composite scenario: built from common industry patterns; the figures are illustrative.
What this covers
- The Situation
- The Problem
- Constraints
- Options Considered
- The Solution
- Implementation
- Results
- What Went Wrong Along the Way
- Lessons Learned
- Apply It Yourself
Key numbers
| Metric | Before | After |
|---|---|---|
| Monthly compute bill (CI + batch) | $23,200 | $6,800 |
| Cost reduction | 0% | 71% |
| CI job failure rate | 0.10% | 0.18% |
| P99 CI job latency | 4 min 12 sec | 4 min 51 sec (+15%) |
| Batch interruption recovery time | ~45 min (full restart) | ~2 min (checkpoint resume) |
Key components
- Karpenter NodePool with diversified instance types — Provisions Spot nodes on demand from a pool of 7+ instance families across 3 AZs, minimising interruption risk and enabling scale-to-zero.
- Actions Runner Controller (ARC) — Scales GitHub Actions self-hosted runners from zero to N pods based on job queue depth, eliminating the always-on runner fleet.
- S3-backed checkpoint library — Allows batch tasks to save progress offsets periodically so an interrupted Spot worker resumes from the last checkpoint rather than restarting.
- SQS Job Queue — Decouples Airflow task submission from worker execution, enabling safe retries and fan-out without losing work items on interruption.
- Prometheus and Alertmanager — Track Spot interruption rates in real time and trigger automated fallback to On-Demand when interruption frequency exceeds safe thresholds.
The Situation
This is a composite scenario assembled from patterns observed across multiple engineering teams; all figures are illustrative and chosen for internal consistency, not to represent any single organisation.
The organisation in question is a 120-engineer B2B SaaS company building a data-integration platform. Their product ingests, transforms, and delivers customer data pipelines, so the engineering team runs a disproportionately large amount of compute: roughly 4,000 CI job-minutes per day across 35 active repositories, plus nightly batch ETL jobs that reprocess customer datasets for reporting. The stack is AWS-native: EKS for workloads, GitHub Actions for CI (with self-hosted runners), Apache Airflow on Kubernetes for batch orchestration, S3 for artifact and checkpoint storage, and RDS PostgreSQL for metadata. The platform team — four engineers — owned infrastructure, cost, and reliability.
The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.
Topics: spot-instances, ci-cd, cost-optimization, kubernetes, infrastructure, batch-processing
Related case studies
- How 37signals Saved $10M by Leaving AWS for Owned Hardware
- How a 120-Engineer SaaS Company Migrated from One AWS Account to a Multi-Account Landing Zone
- How a B2B SaaS Team Cut Their AWS Bill by 40% with Rightsizing, Graviton, Savings Plans and S3 Lifecycle Policies
- How a B2B SaaS Team Split a 4,000-Resource Terraform Monolith Into Modules Without Downtime
- Lift-and-Shift to AWS First, Then Modernise: How a B2B SaaS Company Escaped the Data Centre
- The 2017 Amazon S3 Outage: One Mistyped Command and the Cost of a Slow Restart