AWS FinOps: Real-Time Cost Monitoring & Optimization Guide
One of the biggest challenges in managing resources on AWS is understanding when costs are incurred versus when they are reported. We typically track costs via AWS Cost Explorer, but this data can have delays of up to 24 hours.
In this post, we will explore an event-driven architecture that allows you to monitor your costs in real-time, overcoming this delay.
Why Real-Time Monitoring?
AWS Cost Explorer is a great tool, but it offers a "reactive" approach. That is, you see the cost of a resource the day after it is created. If a very expensive EC2 instance (e.g., p3.16xlarge) is launched accidentally, you might have already spent thousands of dollars by the time you notice.
With real-time monitoring (a proactive approach), you can be notified the moment a resource is created, preventing budget overruns before they happen.
Solution Architecture
We will use 5 core AWS services in this solution:
- CloudTrail: Captures API calls (e.g.,
RunInstances). - EventBridge: Filters events from CloudTrail and routes relevant ones.
- Lambda: Analyzes the event, calculates the cost using the AWS Pricing API, and generates an alert if it exceeds a threshold.
- SNS (Simple Notification Service): Delivers alerts to you via email or SMS.
- IAM: Ensures secure communication between services.
Thanks to this architecture, you receive an alert within seconds, for example, when someone launches an expensive server.
Step-by-Step Setup Guide
Below you can find the steps required to set up this system.
Step 1: Enable CloudTrail
First, you must create a CloudTrail trail that records all API calls made in your AWS account. This trail will capture critical events like RunInstances and forward them to EventBridge.
Step 2: Create an EventBridge Rule
EventBridge allows us to select only the events we are interested in from thousands of CloudTrail events. For example, you can use the following JSON pattern to capture only EC2 instance launch events:
{
"source": ["aws.ec2"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["ec2.amazonaws.com"],
"eventName": ["RunInstances"]
}
}
This rule is triggered only when a RunInstances call is made, avoiding unnecessary Lambda costs.
Step 3: Lambda and Pricing Check
When the EventBridge rule is triggered, we run a Lambda function. This function takes the properties of the created resource (e.g., t3.micro or m5.large) and queries the AWS Pricing API to find its hourly cost.
An example pricing query (via CLI):
aws pricing get-products --service-code AmazonEC2 --filters "Type=TERM_MATCH,Field=instanceType,Value=t3.micro" --region us-east-1
Your Lambda function performs this query using the SDK and generates an alert if the cost exceeds a threshold you set (e.g., $1 per hour).
Step 4: Receive Notifications via SNS
Finally, create an SNS Topic to deliver the alert generated by your Lambda function to you and subscribe your email address. Lambda sends a message like "Warning: An expensive resource (m5.large) has been created!" to this SNS topic.
Frequently Asked Questions (FAQ)
1. What is the cost of this system?
Since this architecture is completely Serverless, you only pay when an event occurs (when a resource is created). It incurs no cost when idle.
2. Does it apply only to EC2?
No. You can monitor creation events for other services like RDS, Lambda, DynamoDB (e.g., CreateDBInstance) via CloudTrail and EventBridge as well.
3. Should I use this instead of Cost Explorer?
No, Cost Explorer is necessary for seeing general trends and historical analysis. This system is a complementary solution for immediate intervention and anomaly detection.
Conclusion
Cost management (FinOps) is not just about paying bills but using resources efficiently. With the event-driven architecture in this guide, you can gain full control over your AWS spending and avoid surprise bills.
For more information, check out our AWS Consultancy and Kubernetes Consultancy services. You can also visit our Home Page for our general tech blog.
Image Suggestions:
- Architecture Diagram: A diagram showing the flow CloudTrail -> EventBridge -> Lambda -> SNS. (Alt text: AWS Event-Driven Cost Monitoring Architecture)
- EventBridge Rule: Screenshot of the rule creation screen in the EventBridge console. (Alt text: EventBridge Rule Configuration for EC2 Events)
Recommended External Links:
Source: https://awsfundamentals.com/blog/aws-finops-realtime-monitoring