Cutting Our Kubernetes Bill: A Practical Playbook

Kubernetes makes it easy to scale up, and just as easy to quietly burn money. Most clusters I've audited run at 20–35% actual utilization while paying for 100%. Here's the playbook I use, in the order that delivers results fastest.

Step 1: Measure before you touch anything

You need per-workload visibility first. Install an open-source cost tool (OpenCost or similar) or use your cloud's native tooling, and answer:

• Which namespaces/teams spend the most?
• What's the gap between requested and actually used CPU/memory?

That requests-vs-usage gap is almost always your biggest win.

Step 2: Right-size requests

Engineers set resource requests defensively, then never revisit them. A service asking for 2 CPUs while using 200m wastes 90% of its reservation, and the scheduler packs nodes based on requests, not usage.

• Pull actual P95 usage from your metrics for every deployment.
• Set requests near P95 usage, limits with sane headroom.
• Automate it: a vertical pod autoscaler in recommendation mode gives you the numbers even if you apply them manually.

This step alone typically cuts 30–40% off the bill.

Step 3: Spot/preemptible instances for everything stateless

Spot capacity is 60–90% cheaper. The rules of engagement:

• Stateless, replicated services: spot by default.
• Batch jobs and CI runners: absolutely spot.
• Databases, queues, anything with local state: on-demand.

Mix node pools and use taints/tolerations so critical pods never land on spot nodes. Modern autoscalers (Karpenter on AWS, GKE Autopilot's flexible provisioning) handle interruptions gracefully enough that this is now boring technology.

Step 4: Autoscale in both directions

• Horizontal pod autoscaling on real signals (requests per second, queue depth), not just CPU.
• Cluster autoscaler / Karpenter to shed empty nodes fast. Check your scale-down settings; the defaults are conservative and leave nodes idling for long windows.
• Scale to zero for dev/staging environments outside working hours. A cron job that scales staging down at night pays for someone's salary in a mid-sized org.

Step 5: Commit on what's left

Only after right-sizing: buy Savings Plans / Committed Use Discounts for your now-honest baseline. Committing before optimizing locks in your waste.

Keeping it optimized

Cost regressions are like performance regressions, and they creep back without guardrails:

1. Cost allocation labels enforced by admission policy.
2. A monthly cost review per team, with the requests-vs-usage gap on one dashboard.
3. Budget alerts that page the owning team, not the platform team.

FinOps isn't a project, it's a practice. The goal isn't the lowest possible bill; it's knowing that every dollar you spend is one you chose to spend.