When Lambda Got Expensive: Moving a High-Throughput API from Lambda to ECS Fargate
A B2B SaaS company's Lambda-based API hit cold-start and cost ceilings at scale; migrating to ECS Fargate cut monthly compute costs by 61% and p99 latency by 74%.
Architecture · intermediate · 12 min read · 2388 words · By Sandip Das · Published
Composite scenario: built from common industry patterns; the figures are illustrative.
What this covers
- The Situation
- The Problem
- Constraints
- Options Considered
- The Solution
- Implementation
- Results
- What Went Wrong Along the Way
- Lessons Learned
- Apply It Yourself
Key numbers
| Metric | Before | After |
|---|---|---|
| Monthly compute cost | $51,000 | $19,800 |
| p99 API latency at peak | 3,800 ms | 980 ms |
| HTTP 500 error rate | 0.40% | 0.02% |
| Aurora connections at peak | ~2,400 (saturated) | ~100 (controlled pool) |
| SLA breach tickets per week | 47 | 3 |
Key components
- ECS Fargate — Runs the containerised Express API as long-lived tasks, eliminating cold starts and enabling fixed-size database connection pools.
- Network Load Balancer — Provides low-latency TCP load balancing to Fargate tasks and integrates with API Gateway via a VPC link.
- API Gateway VPC Link — Preserves existing public endpoint URLs for partner integrations while routing traffic to the new Fargate backend.
- RDS Proxy with pg-pool — Combines RDS Proxy connection multiplexing with a fixed per-task connection pool to keep Aurora connection counts predictable under autoscaling.
- ECS Application Auto Scaling — Scales Fargate task count based on CPU utilisation with asymmetric cooldowns to handle traffic spikes without thrashing.
- API Gateway Canary Deployments — Enabled incremental traffic shifting from Lambda to Fargate with instant rollback capability at each percentage step.
The Situation
This is a composite scenario assembled from patterns observed across multiple real-world migrations; all figures are illustrative and chosen for internal consistency, not to represent any single company.
The organisation is a 120-engineer B2B SaaS company selling a workflow-automation platform to mid-market and enterprise customers. Their platform exposes a REST API consumed by a React web app, native mobile clients, and roughly 40 partner integrations. The backend team of 14 engineers owned the API layer, built entirely on AWS Lambda behind API Gateway. The stack had served them well from 0 to about 8 million API calls per day, but by the time the platform reached 35 million calls per day the cracks were showing in both the AWS bill and in Datadog dashboards.
The core tech stack at the time of the incident: Node.js 18 Lambda functions (average 512 MB memory, 15-second timeout), API Gateway (REST API type), Aurora PostgreSQL 14 via RDS Proxy, ElastiCache Redis 7 for session and rate-limit state, S3 for artifact storage, and SQS/SNS for async fan-out. CI/CD ran on GitHub Actions pushing through AWS CodePipeline into CloudFormation stacks.
The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.
Topics: aws, serverless, ecs, lambda, cost-optimization, architecture