Cloud FinOps: How We Cut AWS Infrastructure Bills by 42% Without Sacrificing Latency

As digital platforms scale to handle millions of daily API transactions, cloud infrastructure spend frequently grows exponentially faster than top-line revenue. For a fast-growing B2B SaaS client running on AWS EKS, monthly cloud costs surged past ₹48,000/month. Within 60 days, our DevOps architects slashed that to ₹27,800/month while cutting P99 API response times by 18%.

1. Initial Infrastructure Audit & Waste Analysis

Our team performed a comprehensive telemetry review using AWS Cost Explorer, Kubecost, and CloudWatch metrics. The audit uncovered four primary cost drivers:

• Overprovisioned EC2 Auto Scaling Groups: Standard Cluster Autoscaler provisioned rigid m5.2xlarge instances, resulting in an average cluster CPU utilization of only 23%.
• Legacy x86 Architecture: Workloads ran exclusively on older Intel/AMD instances with lower price-performance ratios.
• Unused On-Demand Workloads: Batch asynchronous workers, CI/CD runners, and background queue processors were running on expensive On-Demand instances rather than discounted Spot instances.
• Unoptimized Storage Footprint: Thousands of unattached legacy EBS gp2 volumes and un-tiered S3 media buckets holding TBs of historical assets.

2. Karpenter Dynamic Kubernetes Node Autoscaling

Traditional Kubernetes Cluster Autoscaler relies on AWS Auto Scaling Groups (ASGs), taking 4–6 minutes to boot homogeneous node sizes. We migrated the EKS cluster to Karpenter, AWS’s open-source, just-in-time node provisioner.

Karpenter directly provisions heterogeneous EC2 instances based on precise pending pod requests. When pods scale down, Karpenter automatically initiates Node Consolidation, gracefully draining and terminating under-utilized nodes in seconds.

3. Seamless ARM64 Graviton3 Migration

AWS Graviton3 (c7g, m7g, r7g) processors deliver up to 25% better compute performance and 20% lower hourly costs compared to equivalent 6th generation x86 instances.

Because the client’s microservices were containerized in Go, Node.js 20, and Python 3.12, we configured Docker multi-arch build pipelines using docker buildx to generate dual linux/amd64 and linux/arm64 images seamlessly:

• Zero Code Refactoring: High-level languages compiled natively for ARM64 without breaking dependencies.
• Immediate 20% Cost Drop: Swapping m5.xlarge (₹0.192/hr) to m7g.xlarge (₹0.163/hr) delivered instant operational savings.
• Cache Performance Surge: Migrated self-managed Redis clusters to r7g.large, reducing memory latency by 14%.

4. Fault-Tolerant Spot Orchestration

Spot instances provide up to 70–90% discounts compared to On-Demand rates, but risk 2-minute termination notices. We implemented a resilient dual-tier compute strategy:

1. Critical API Gateway & Core State Services: Maintained on Reserved Instances (RIs) and 3-Year Savings Plans for predictable baseline coverage.
2. Background Jobs, Webhook Processors & Analytics: Routed 100% to Karpenter Spot pools with capacity-optimized-prioritized allocation and AWS Node Termination Handler.

5. S3 & EBS Storage Lifecycle Optimization

Storage waste is the most insidious silent budget drain. Our automated remediation scripts executed:

• gp2 to gp3 Migration: Converted 80+ EBS volumes to gp3, delivering 20% cost savings and guaranteed 3,000 IOPS baseline without paying for provisioned volume size.
• S3 Intelligent-Tiering: Configured automated lifecycle policies shifting objects unaccessed for 90 days directly to Glacier Instant Retrieval.

6. Production Karpenter NodePool Manifest

Here is the exact production-ready Karpenter configuration enabling Graviton ARM64, Spot fallbacks, and aggressive node consolidation:

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: finops-optimized-general
spec:
  template:
    spec:
      requirements:
        # Prefer energy-efficient ARM64 Graviton instances
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64", "amd64"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["c7g", "m7g", "t4g", "c6i", "m6i"]
        - key: karpenter.k8s.aws/instance-size
          operator: In
          values: ["large", "xlarge", "2xlarge"]
      nodeClassRef:
        name: default-ec2-node-class
  # Automatically consolidate underutilized nodes
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h # 30 days node refresh
  limits:
    cpu: 250
    memory: 500Gi

7. Financial Breakdown & Verified Results

The financial results verified over a 90-day post-migration observation period:

• Monthly AWS Spend: Reduced from ₹48,200 to ₹27,800 (-42.3%).
• Cluster CPU Utilization: Rose from 23% to 68% through optimal container bin-packing.
• P99 API Latency: Improved by 18ms due to Graviton3 cache bandwidth improvements.
• Zero Incident Downtime: 100% SLA uptime achieved across all Spot instance rebalances.