The $30,000 NAT Gateway Bill: Finding the Traffic and Fixing It with VPC Endpoints

A B2B SaaS company discovered half its AWS bill was NAT Gateway data charges from S3 and DynamoDB traffic. VPC endpoints eliminated the cost in two weeks.

Networking & APIs · beginner · 12 min read · 2368 words · By Sandip Das · Published

What this covers

  1. The Situation
  2. The Problem
  3. Constraints
  4. Options Considered
  5. The Solution
  6. Implementation
  7. Results
  8. What Went Wrong Along the Way
  9. Lessons Learned
  10. Apply It Yourself

Key numbers

MetricBeforeAfter
Monthly NAT Gateway data charge$28,400$1,260
NAT Gateway data volume68 TB/month3.1 TB/month
Total monthly networking cost$30,000$3,280
Application code changes requiredN/A0
Annualised infrastructure saving$0~$320,000

Key components

  • S3 Gateway Endpoint — Routes S3 traffic from private subnets directly to S3 over the AWS backbone at zero data-processing cost, bypassing the NAT Gateway entirely.
  • DynamoDB Gateway Endpoint — Routes DynamoDB traffic from private subnets directly to DynamoDB over the AWS backbone at zero data-processing cost.
  • CloudWatch Logs Interface Endpoint — Provides a private ENI inside the VPC for CloudWatch Logs traffic, eliminating 4 TB/month of NAT Gateway data charges at 78% lower per-GB cost.
  • ECR Interface Endpoint — Allows EKS nodes to pull container images without traversing the NAT Gateway, improving pull latency and eliminating associated data charges.
  • VPC Flow Logs + Athena — Provides byte-level attribution of which destination IPs are consuming NAT Gateway bandwidth, enabling data-driven prioritisation of endpoint creation.
  • AWS Cost Explorer usage-type breakdown — Surfaces NatGateway-Bytes as a distinct line item, making the cost visible and measurable before and after the fix.

The Situation

The organisation in question is a 120-engineer B2B SaaS company building a data-enrichment platform. Customers pipe CRM records into the platform, a fleet of workers normalises and scores them using an ML model, and the enriched records are written back via a REST API. The stack is almost entirely AWS: EKS for compute, S3 for object storage, DynamoDB for job-state tracking, ECR for container images, and CloudWatch Logs for observability. All workloads run in private subnets in a single us-east-1 VPC. A pair of NAT Gateways — one per availability zone — provides outbound internet access for the private subnets.

At the time of the incident the platform was processing roughly 40 million records per day. The engineering team was split into five product squads and a small platform squad of four engineers. Infrastructure was managed with Terraform. There was no dedicated FinOps function; cost reviews happened quarterly, not monthly.

The rest of this case study — the trade-offs, the scaling numbers and the build prompt — is available to LearnXops members. Create an account or see membership.

Topics: aws, networking, cost optimisation, vpc, s3, infrastructure

Related case studies

Browse all case studies