How a B2B SaaS Platform Raised CDN Cache Hit Ratio from 40% to 92% and Halved Origin Load
A mid-size SaaS company fixed broken cache-control headers, split cacheable from dynamic traffic, and introduced edge-side cache warming to cut origin load by 52% and slash p99 latency.
Caching · intermediate · 12 min read · 2374 words · By Sandip Das · Published
Composite scenario: built from common industry patterns; the figures are illustrative.
What this covers
- The Situation
- The Problem
- Constraints
- Options Considered
- The Solution
- Implementation
- Results
- What Went Wrong Along the Way
- Lessons Learned
- Apply It Yourself
Key numbers
| Metric | Before | After |
|---|---|---|
| CDN cache hit ratio | 40% | 92% |
| Peak origin requests per hour | 1,200,000 | 576,000 |
| p99 API latency | 1,840 ms | 610 ms |
| API cluster monthly EC2 cost | ~$4,800 | ~$2,400 |
| Dashboard median load time | 3.1 s | 1.2 s |
Key components
- CloudFront Cache Behaviours — Split static, public-API, and private-API traffic into separate cache policies, eliminating the catch-all behaviour that was busting the cache.
- FastAPI Cache Header Middleware — Centrally sets correct Cache-Control and tenant-scoped cache key headers on every API response, replacing the missing or incorrect per-route defaults.
- CloudFront Cache Policy (custom key) — Uses a hashed X-User-Tenant header as the cache key differentiator instead of raw cookies, providing tenant isolation without forwarding session tokens.
- Cache Warmer Lambda — Pre-populates CloudFront edge caches with the top-200 paths after every production deploy, eliminating the cold-start latency spike.
- Per-Behaviour Grafana Dashboard — Surfaces cache hit ratio, origin latency, and byte hit ratio broken down by CloudFront behaviour, closing the observability gap that hid the problem for years.
- Redis Read-Through Cache — Acts as a second caching layer inside the API tier, protecting PostgreSQL from cold-cache origin requests that CloudFront cannot serve.
The Situation
This is a composite scenario assembled from patterns observed across multiple production environments; all figures are illustrative and chosen for internal consistency, not to represent any single organisation.
The organisation is a 120-engineer B2B SaaS company selling a project-management and reporting platform to mid-market customers. Their product serves roughly 85,000 daily active users across North America and Western Europe. The engineering team is split into six product squads and a three-person platform team. The stack is a React single-page application backed by a Python/FastAPI API tier, PostgreSQL 14 for transactional data, Redis 7 for application-level caching, and AWS as the cloud provider. Traffic is fronted by CloudFront with an ALB behind it. At the time of this case, the platform team had just taken ownership of CDN configuration after it had been set up years earlier by a contractor and never revisited.
The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.
Topics: cdn, caching, performance, nginx, cloudfront, infrastructure