Why ECS could not pack and what moving to EKS did to compute costs

Industry: Media and events
Company size: 200-500 employees
Services: Platform engineering, Kubernetes, cloud cost optimization, migrations

The Challenge

When this work began in 2025 the client ran everything on ECS: 84 services across Fargate and a small EC2 cluster, plus a scattering of EC2 boxes and SaaS subscriptions for the tooling around them. On ECS you pay for what you reserve, and reservations were generous. Most services had been deployed at one vCPU and two gigabytes because that was the template, workers were copied from the web task, and nobody shrank a task once it was running, because on Fargate a smaller task means less headroom and a redeploy with no easy way back.

The other half of the problem is that ECS cannot pack. Fargate bills each task's reservation in isolation, and on the EC2 launch type the scheduler places by reservation, so hosts fill up on paper long before they do any work. We measured it. The remaining production fleet reserves 57 vCPU and uses about 1. The EC2 container hosts show 34 percent CPU reserved and 1 percent used, with memory at 37 percent reserved and 21 percent used. Fargate tasks in public subnets carried 89 public IPv4 addresses at a monthly cost, and Fargate's round-robin placement across zones made ECS the largest single source of cross-zone data transfer charges.

Our Approach

The platform work is covered in the GitOps case study. This is the capacity side of it: how the same workloads ended up on a fraction of the compute, and what it took to keep them there. The cluster stood up in spring 2025, production Karpenter pools landed that August, spot became the default that December, the first production application cut over in May 2026, the whole Laravel fleet moved to spot in June, and the rebuilt flagship site launched in August. Migration is still in progress.

• Requests that mean something. Every workload on EKS declares a CPU and memory request based on measured usage, not on the template it was born with. Goldilocks and the vertical pod autoscaler produce the recommendations; a human applies them. A request is the unit the scheduler packs on, so this is the step that turns over-provisioned tasks into dense nodes.
• Karpenter chooses the node, not a person. Node pools describe what is allowed: instance families, architectures, spot or on-demand, zones. Karpenter launches the cheapest instance that fits the pending pods and consolidates when a node is empty or underutilized, after five minutes on the spot pools and thirty on the pools that hold databases. Nothing on the cluster is a fixed-size host that someone sized by hand.
• Spot by default, with a graceful exit. Web and worker pods for eleven Laravel applications moved to spot capacity through one change to the shared Helm chart, which also added the preStop delay and termination grace that let the load balancer drain a pod before it goes. Spot is 65 to 70 percent cheaper than on-demand. Interruption-intolerant work, mostly databases, pins to an on-demand pool that is deliberately left untainted so ordinary pods can fill its spare capacity.
• Savings plans sized to the on-demand floor. Spot is not covered by compute savings plans, so the plans are sized to the on-demand baseline and run at 100 percent utilization. Adding spot nodes does not add plan waste.
• A right-sizing pass when the cluster drifted. In June the production cluster had grown to 26 nodes after a rollout burst, most of them small on-demand instances sitting mostly idle, with CPU requests running about six times actual usage. Raising the instance floor to fewer, larger nodes, widening the consolidation budget from 5 to 20 percent, and trimming the worst requests reclaimed nine cores in an afternoon.

The Results

Measured on the production cluster and the remaining ECS fleet on September 23, 2026, with billing from the organization's cost explorer, which holds fourteen months of history, so the baseline is July 2025, just before the production node pools went in.

• Twenty nodes run 522 pods and 84 application workloads. Eight of the twenty are spot. Nodes routinely sit at 65 to 89 percent memory, which is what packing looks like.
• Over ten times the work per compute dollar. In August the ECS line and the EC2 line were within a few hundred dollars of each other. One delivered about one vCPU of actual work; the other delivered nineteen.
• Fargate cross-zone transfer dropped from $939 in May to $77 in the first half of June as the heaviest workers left ECS. Some of that cost reappears on the EKS side, where it can be fixed with zone-aware routing rather than tolerated.
• The internal platform rode along on spare compute. Grafana, Loki, and Prometheus, Sentry, the Harbor registry, self-hosted CI runners, Vitess, an automation platform, and the AI sandbox all run on the same clusters. On production, internal platform services account for about 40 percent of requested CPU, and none of them has its own line on the bill. Before, each would have meant its own hosts or its own SaaS invoice.
• The total bill did not fall, and that is the honest version. Over the fourteen months the cost explorer can see, the EKS build-out added about $1,000 a month net: EC2 up, ECS down, and that is the whole price of hosting the migrated applications, the rebuilt flagship site, a Postgres data platform, and every internal service listed above. Month-to-month spend stayed roughly flat because data and ETL growth elsewhere ate the difference. The win is capacity per dollar. The 55 ECS services still to migrate, reserving 57 vCPU to use one, are the savings that have not been collected yet.

What did not go to plan

• The vertical autoscaler and the horizontal autoscaler must not both act on CPU. On launch day of the new site the VPA rewrote a web pod's request from 400 millicores to 23, the HPA divided by 23, and the fleet flapped between 6 and 12 pods on no load. The VPA now recommends only for anything the HPA scales.
• Balloon pods, the usual trick for keeping warm capacity, do not work on Karpenter. It provisions a fresh node faster than the scheduler can evict the balloon, so the real pod lands cold every time. Warm capacity has to be real, scheduled, and paid for.
• Spot is a default, not a rule. Etcd, a Postgres writer mid-load, Harbor's garbage collector, and a search index rebuild each got moved to on-demand after an interruption cost more than the discount saved. The on-demand pool exists for exactly those.
• Consolidation does not clean up after a rollout by itself. A 5 percent disruption budget on a 26-node cluster is one node at a time, which is why the June pass needed a bigger budget to reclaim what the rollout had spread out.

Key Takeaway

Over-provisioning on ECS is not a sizing mistake somebody made. It is what the platform makes easy, because nothing ever asks whether a task's reservation matches its use. Kubernetes asks on every schedule, Karpenter answers with the cheapest node that fits, and the cost of an idle vCPU finally becomes visible enough to fix.

Related guide: ECS vs EKS: when moving is worth it, and what it actually saves

Paying for reservations you never use?

We'll pull your ECS reservation and utilization numbers for the last two weeks and show you what the same fleet would need on a packed cluster.

Schedule a call