How a B2B SaaS Team Split a 4,000-Resource Terraform Monolith Into Modules Without Downtime
A 120-engineer SaaS company decomposed a single 4,000-resource Terraform state file into scoped modules and isolated state backends, cutting plan times from 22 minutes to under 90 seconds with zero production incidents.
Infrastructure · advanced · 12 min read · 2225 words · By Sandip Das · Published
Composite scenario: built from common industry patterns; the figures are illustrative.
What this covers
- The Situation
- The Problem
- Constraints
- Options Considered
- The Solution
- Implementation
- Results
- What Went Wrong Along the Way
- Lessons Learned
- Apply It Yourself
Key numbers
| Metric | Before | After |
|---|---|---|
| terraform plan duration (largest domain) | 22 minutes | 88 seconds |
| deployments per week | 4–5 | 14–16 |
| mean time to apply a hotfix | 2 hr 10 min | 22 minutes |
| lock contention incidents per month | 11 | 0 |
| engineer onboarding to first independent apply | 60 days | 8 days |
Key components
- terraform state mv — Surgically relocates resource records between state files without destroying or recreating the underlying AWS resources.
- S3 + DynamoDB per-domain backends — Provides isolated locking and versioned state storage for each infrastructure domain, eliminating cross-team lock contention.
- terraform_remote_state data sources — Allows downstream domains to read outputs from upstream domains without collapsing them back into a single state file.
- Private Terraform Module Registry — Centralises reusable infrastructure patterns (VPC layouts, EKS node groups) as versioned, pinnable modules consumed across all domain roots.
- Parallel CI plan jobs — GitHub Actions runs plan jobs for all four domains simultaneously, replacing the serialised monolith pipeline and recovering the majority of CI wait time.
- Zero-change plan gate — Requiring both source and destination roots to show zero planned changes after each state mv batch is the primary safety mechanism preventing accidental resource destruction.
The Situation
This is a composite scenario assembled from patterns the author has observed across multiple organisations; all figures are illustrative and internally consistent, not drawn from any single company's incident reports.
The organisation is a 120-engineer B2B SaaS company running a multi-tenant analytics platform on AWS. The platform serves roughly 800 enterprise customers and processes around 40 TB of event data per day. The infrastructure team — six engineers — owns everything from VPC topology to EKS node pools to RDS Aurora clusters to IAM policy sets. All of it, every last resource, lived in a single Terraform root module backed by one S3 state file and one DynamoDB lock table. At the time the migration began, terraform state list returned 4,127 resources.
The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.
Topics: terraform, infrastructure-as-code, state-management, platform-engineering, aws, devops
Related case studies
- How 37signals Saved $10M by Leaving AWS for Owned Hardware
- How a 120-Engineer SaaS Company Migrated from One AWS Account to a Multi-Account Landing Zone
- How a B2B SaaS Team Cut CI and Batch Costs by 70% Using Spot Instances Without Losing Jobs
- How a B2B SaaS Team Cut Their AWS Bill by 40% with Rightsizing, Graviton, Savings Plans and S3 Lifecycle Policies
- Lift-and-Shift to AWS First, Then Modernise: How a B2B SaaS Company Escaped the Data Centre
- The 2017 Amazon S3 Outage: One Mistyped Command and the Cost of a Slow Restart