Loading...
Back to Services

Case Study

AWS Cost Review & Optimization Automation

FinOps Automation Python boto3

Cost data → Audit → Evidence → Savings

FinOps · Jun 2026  ·  ~9 min read  ·  By Tung Le

AWS Cost Review & Optimization Automation — From 2 Days to 30 Minutes

Summary: A manual AWS cost review used to take 1–2 days and depended on the reviewer's experience. I built an automated cost review system in Python — it collects billing data, inventories resources, detects waste against a 7-pillar framework, cross-checks evidence, and produces a report. Results on a real engagement: 90% less time, $248–360/month in potential savings identified, and every recommendation backed by concrete cost evidence.

-90%
Review time
(~18–24h → ~30 min)
$248–360
Potential savings
per month
+33.8%
Actual spend vs
internal estimate
7
Review pillars
standardized

Technology

Pythonboto3AWS CLI v2 Cost Explorer APICompute OptimizerTrusted Advisor AWS BudgetsCost Anomaly Detection EC2 / EBSVPC EndpointsSavings Plans / RI

My Contribution

Designed and built the entire automated AWS cost review system myself: cost data collection, resource inventory, waste detection, evidence cross-checking, savings estimation, and client report generation.

Step 1 — The problem: manual review doesn't scale

Before building this system, each AWS cost review for a client took 1–2 days of manual work. The process usually went:

  • Open AWS's cost analysis tool, filter by each service, take screenshots
  • Log into the console and manually inspect every resource type (volumes, snapshots, network endpoints...)
  • Compile everything into a spreadsheet and write the report

The issues: reliance on the reviewer's experience, no consistent standard, and hard to scale across clients — every review started from scratch, with no guarantee of full, even coverage.

Case anonymized — all absolute figures and client resource names are hidden.

Step 2 — Building the automated system (Python + AI)

I designed the workflow around 3 principles:

  1. Fixed framework, changing data — always review against the same 7 pillars and a consistent way of classifying waste
  2. Machines do the repetitive work, humans do the judgment — scripts collect & compute automatically; humans verify and make the call
  3. Evidence, not guesswork — every conclusion traces back to a specific data source

Input

Provided by the client
Project info — client, analysis period, AWS region
Kickoff questionnaire — system architecture, assumptions
Architecture diagram — to map costs against the system

Automated processing

Python — runs directly, no infrastructure to set up
Collect monthly bills from Cost Explorer
Analyze: biggest cost drivers, how far actual differs from estimate, which months spiked
Scan for waste: volumes (EBS), snapshots, network endpoints, long-stopped servers
Check for over-provisioned resources (via Compute Optimizer)
Review utilization of prepaid commitments (Savings Plans / Reserved Instances)
Check cost governance tooling: budget alerts, anomaly detection

AI interpretation & drafting

Speeds up the most time-consuming part — but doesn't decide
Summarize raw output for quick reading
Explain each finding: why it's wasteful and how to fix it
Suggest suspicious candidates from the inventory data
Generate a draft client report

Humans verify & decide

What AI and the machine are not allowed to do alone
Verify AI recommendations, eliminate false positives
Cross-check resource relationships before recommending (avoid accidental deletion)
Review the draft report before it reaches the client
Finalize recommendations & roadmap

Result

Delivered to the client
List of findings — with cost impact & priority
Summary report for the client

The full workflow — Python handles data, AI handles interpretation & drafting, humans keep the decisions.

Technical detail: the pipeline modules For engineers

The pipeline is written in Python 3.10+ (standard library + boto3 only), running directly through AWS CLI v2 under the client's profile (auth via SSO / saml2aws) — no dedicated infrastructure needed.

ModuleResponsibility
fetch_costs.pyPull Cost Explorer bills by month (Unblended Cost), grouped by SERVICE / USAGE_TYPE
analyze_costs.pyPareto of top services, architecture grouping, Estimate vs Actual, month-over-month anomalies (rule-based)
investigate_findings.pyScan EBS / snapshot / VPC Endpoint / EIP / ALB for orphaned or idle resources (evidence-based, with traffic checks)
investigate_provisioning.pyRight-sizing via Compute Optimizer
investigate_commitments.pySavings Plans / Reserved Instances utilization
investigate_finops_tools.pyCheck Budgets, Cost Anomaly Detection, Trusted Advisor, Cost Allocation Tags
drill_costs.pyDeep-drill by USAGE_TYPE, compute run-rate around an event/cutover

Configured via engagement.json (client, analysis period, region scope, auth profile). Outputs: run-summary.md + investigation-summary.md (raw data, no narrative yet) for the reviewer to read before writing findings.

Step 3 — The 7-pillar review framework

Every engagement is reviewed against the same 7 pillars, so nothing gets missed:

Pillar Description Example findings
1. Unused resources Resources still existing but serving nothing Orphaned volumes, extra snapshots, unused network endpoints
2. Right-sizing Resources provisioned larger than actual need Over-provisioned servers and databases
3. Prepaid commitments Making use of long-term committed pricing Commitment plans under-utilized or missing
4. Storage lifecycle Managing old data to cut storage cost Storage with no policy to clean up old data
5. Data transfer Cost of moving data within/out of AWS NAT gateways, cross-region transfer
6. Operational scheduling Turning resources off/on by working hours Test environments running 24/7 unnecessarily
7. Cost governance Controls & alerting for cost Budget alerts, tagging, an owner reviewing bills

Step 4 — Pilot on a real client

Applied to the first engagement — a production AWS environment in a US region (3 months of billing analyzed):

Metric Value
Average spend / month Hidden for confidentiality
Actual spend vs client's estimate 33.8% higher
Savings opportunities identified $248–360/month
Cost governance maturity 1.5/5 (internal scale)

Cost breakdown by architecture group (3 months combined):

Group % of total bill
Compute83.5%
Security & logging9.6%
Network & CDN3.4%
Data & storage2.9%
Other0.6%

Figures anonymized — only percentages shown, no absolute amounts or client resource names.

Step 5 — Key findings

EBS volumes — $218–300/month in savings

  • Finding: 27 problematic volumes — orphaned ones (attached to nothing), and production databases still on the older, pricier volume type (gp2) instead of the newer gp3 at the same performance.
  • Recommendation: Delete orphaned volumes, migrate databases to the cheaper newer volume type.
Technical detail: detection & gp2 → gp3 For engineers

How wasteful volumes are detected (evidence-based):

  • Orphan: volume in available state (no attachment) — not attached to any instance.
  • Attached-to-stopped: volume in-use but the target instance is stopped — still billing for storage even though nothing runs.
  • Cross-checked against Compute Optimizer and Trusted Advisor (idle volumes) to raise confidence before recommending.

gp2 → gp3: gp3 is ~20% cheaper at the same baseline (3,000 IOPS / 125 MB/s free), decoupling capacity from performance. Migrated via modify-volume — no detach, no downtime, but best done in a maintenance window to monitor.

aws ec2 modify-volume --volume-id vol-xxxx \
  --volume-type gp3 --iops 3000 --throughput 125

Savings estimated conservatively — only the certain parts (orphan + gp2/gp3 delta), no optimistic stacking of assumptions.

Long-stopped servers still billing for storage

  • Finding: A common cost trap — a stopped server (EC2) no longer bills for compute, but its attached volumes (and static IP, if any) keep billing continuously. Many machines had been off for months with nobody noticing, because the bill never "complains".
  • Recommendation: For machines stopped long enough → consider deleting the machine and its volumes. For recently stopped ones → confirm with the owner first, to avoid deleting something intentionally paused.
Technical detail: classifying stopped machines for deletion For engineers

The system reviews from an instance-level view (not just per isolated volume): it rolls up total attached EBS + number of static IPs + days stopped into one row, to directly answer "which machine should be removed".

Days stopped is derived from the instance's StateTransitionReason (format User initiated (YYYY-MM-DD ...)) — no extra API call. Three states:

StateConditionAction
Terminate candidateStopped ≥ 30 days (threshold adjustable)Strong deletion candidate — off for long, unlikely still needed
WatchRecently stopped, or stop time unknownConfirm with the owner before acting
Running / other—Skipped, not included

Boundary note: StateTransitionReason only keeps the most recent state change, and doesn't cover things like Capacity Reservations / licenses billed separately while stopped — so still cross-check with the client's input when in doubt.

Network endpoints (VPC Endpoints) — $30–60/month in savings

  • Finding: 10 private connections to AWS services costing ~$128/month, some of them barely used.
  • Recommendation: Keep the essential endpoints, review and remove the rarely used ones.

Cost governance — indirect impact

  • Finding: No budget alerts, no mandatory resource tagging, no one reviewing bills regularly.
  • Recommendation: Enable budget alerts and build a tagging policy to catch abnormal cost increases early.

Step 6 — Every recommendation is backed by evidence

The guiding principle: no cost-cutting proposals based on guesswork. Every recommendation is cross-checked across multiple data sources (billing, AWS advisory tools, actual resource configuration) and its real impact verified before it enters the report.

  • Check the relationships between resources before recommending deletion — e.g. a snapshot may still be backing a machine image — to avoid breaking a running system.
  • Estimate savings conservatively, with specific resource IDs so the engineering team can act immediately.
  • Classify clearly: optimizable / accepted (due to business requirements) / needs further monitoring.
The key point: The machine handles collection & computation; humans handle judgment & verification. The client receives verified recommendations — not a list of "suspicions".
Technical detail: the cross-check logic before recommending deletion For engineers

A classic example: some snapshots initially look "orphaned" because their source volumes were deleted. But if a snapshot is registered by an AMI as a block device mapping, deleting it breaks the AMI — you can no longer launch from it.

Decision rule before flagging a snapshot as waste:

snapshot has no source volume?
        │
        ▼
registered by any AMI?  ── YES ─▶ KEEP (not waste)
        │
        NO
        │
        ▼
has a backup marker (AWS Backup / DLM)? ── YES ─▶ KEEP
        │
        NO
        │
        ▼
   deletion candidate (awaits human confirmation)

The same principle applies to other resources: EIPs check association, ALBs check target-group traffic, VPC Endpoints check actual consumers. Nothing is deleted based on surface state alone.

Results

Metric Before After Change
Time per review 1–2 days ~30 min 90% less
Potential savings — $248–360/month New
Recommendation confidence Depends on experience Cost evidence + resource IDs Standardized
Review coverage Easy to miss things All 7 pillars Consistent

Lessons learned

  1. Automate for speed, keep humans for correctness — the machine handles collection & computation, the expert handles judgment & verification.
  2. A fixed framework prevents blind spots — 7 pillars ensure full, consistent coverage across every engagement.
  3. Every recommendation is backed by evidence — cross-check multiple data sources before proposing.
  4. Governance matters as much as optimization — budget alerts & tagging catch cost spikes early and prevent recurrence.
  5. Measurable and actionable results — conservative savings estimates with concrete actions for the engineering team.

Need an AWS cost review & optimization?

I take on AWS cost reviews — from building an automated process to detailed manual audits, with a concrete savings roadmap.

Get in touch