Get EC2 and RDS reserved instances to reduce costs

[peterdudfield]

Is there a way in terraform to research the ec2 and rds instances. This should bring down costs for us

WARNING: this is a good first issue, if you know terraform

[avreetkaur84]

• Terraform doesn’t automatically “research” EC2 and RDS instances, but it can help analyze and optimize resources over time by managing and updating them.

• Terraform uses a state file to track resource configurations: This allows us to view the current state of our EC2 and RDS instances. By comparing the current state with the desired state, Terraform can highlight under-utilized or over-provisioned resources.

• Data sources in Terraform: We can use Terraform’s data sources to query the AWS infrastructure, such as pulling details about the current EC2 and RDS instances. This helps us gather information like instance sizes, storage, and utilization, which is essential for cost optimization.

• Cost optimization via updates: Once we identify over-provisioned EC2 instances or underutilized RDS databases through Terraform’s state and data sources, we can update the resources to match actual needs (e.g., resizing instances, adjusting storage). Terraform automatically handles these updates, saving time and ensuring we're not wasting resources.

• Identifying under-utilized resources: Using Terraform's state file, we can identify areas where resources can be downsized, such as reducing EC2 instance sizes or scaling down RDS storage, which directly leads to cost savings.

[Clemo97]

@avreetkaur84 and what about this https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_capacity_reservation
and this
https://registry.terraform.io/providers/-/aws/latest/docs/resources/rds_reserved_instance

[jamescrowley]

@peterdudfield guessing you mean reserve not research? Depending how you're managing billing, I'm not sure I'd recommend incorporating making the actual reservations into your terraform. Reserved Instances are automatically applied to running on-demand instances, so once the reservations are made at your account/billing level you wouldn't need to change anything on the Terraform side. Happy to share more if useful.

[peterdudfield]

Thanks @jamescrowley , yea any info on, links how to do this, that would be great

[naveen-vijay]

Is there a way I could learn or lookup what is the current configuration and spend associated with ec2 and RDS ? Knowing what the current spend would help strategize this better. Also I am curious to see if we are able to rearchitect the current setup (if possible) to use spot instance or even better serverless with lambda to take advantage of cost options in the cloud.

So if someone could clarify the below questions - it would help me research and recommend the best ways to cut down the cost.

1. Is the cloud spend or usage of EC2 and RDS - predictable - then reserved instance is a good candidate. We need instance size, tenancy (dedicated host or instance), region/AZ and platform plus the duration we'd like to commit (1 year or 3 years). Followed by the decision on how much we'd want to pay for the Reserved Instance (No, Partial or all upfront). All Upfront gives the best pricing - if we are willing to trade pricing in return for commitment / consistency.

2. If not - then we should consider rearchitecting or before that relooking at the architecture.

I am happy to support either ways. In any case - there is very little scope and involvement of Terraform, if all.

[peterdudfield]

1. Its all in the terraform. Its mainly ec2 instance for elasticbeanstalk which are constantly on. I'm happy to hear either way, and what savings could be amde

[jamescrowley]

@peterdudfield we should definitely be able to reduce costs if they're always on, but in particular given our climate focus and to @naveen-vijay's point, I'd suggest we first review if there was an alternative to running that infra 24/7.

It may be that your elastic beanstalk / terraform config is already set to scale to 0 though. I can see some airflow related usage of elastic beanstalk in the terraform but I'm assuming that's task-based (and so spins down again).

Also ideally if you were able to provide read-only access to the AWS account I'd be happy to review and see what the options might be based on previous usage.

If either of those are worthwhile and feasible, I'd be happy to dig into that further.

[peterdudfield]

Thanks @jamescrowley for this.
I'm not sure I can get you a read only account, there would be some security risks here.

Those Elastic Beasntalk dont scale down, as they need to be ready to go. I.e if somebody wants a forecast, they need it straight away. Perhaps on our dev envionrment we could do this, but not sure to be honest.

The elasticbenstalk apps we have are

• api: national gsp - https://github.com/openclimatefix/ocf-infrastructure/blob/main/terraform/nowcasting/development/main.tf#L92
• api site - https://github.com/openclimatefix/ocf-infrastructure/blob/main/terraform/nowcasting/development/main.tf#L798
• dashboard - https://github.com/openclimatefix/ocf-infrastructure/blob/main/terraform/nowcasting/development/main.tf#L704
• ariflow https://github.com/openclimatefix/ocf-infrastructure/blob/main/terraform/nowcasting/development/main.tf#L769

I think we would be willing to do Partial payment for a year? Im intereted in roughly how much that would save?

[jamescrowley]

@peterdudfield I’ll take a closer look at the TF but assuming you have access, AWS will have generated some recommendations and estimated savings - https://docs.aws.amazon.com/cost-management/latest/userguide/ri-recommendations.html#viewing-rex

Do you know what the cold start time is like? And the idle time vs actually serving requests?

[peterdudfield]

What do you mean by start time? For the API is active every 2 minutes. Some of the others its a bit more idle, but we want the start time to be very little. The airflow needs to be active all the time.

[jamescrowley]

@peterdudfield cold start time is how long it takes from first request to be served up a response (if the environment is spun down to 0). The shorter you can get it (most often in a Lambda world) the more readily you can scale to zero.

The other aspect I'd look at here is utilisation of the provisioned capacity - how often is the traffic peaky, how often is the server over-provisioned? (appreciate that beanstalk will already be doing some of the work here in terms of auto-scale), Or is it pretty steady and predictable? Happy to hop on a call sometime if it would be useful

[peterdudfield]

@peterdudfield cold start time is how long it takes from first request to be served up a response (if the environment is spun down to 0). The shorter you can get it (most often in a Lambda world) the more readily you can scale to zero.

The other aspect I'd look at here is utilisation of the provisioned capacity - how often is the traffic peaky, how often is the server over-provisioned? (appreciate that beanstalk will already be doing some of the work here in terms of auto-scale), Or is it pretty steady and predictable? Happy to hop on a call sometime if it would be useful

I think we just cant scale down to 0, they need to 1, and then they get used every minute.
This is why I think we always have this 1, or several on different eb applications, that we could reduce the cost by having a reserved instance

[jamescrowley]

Sure, understood :) Depending on if you go for a convertible or standard reserved instance it should be 30-40% discount depending on the instance types. Worth checking the recommendations AWS makes in the console. You can also get an estimate here: https://aws.amazon.com/ec2/pricing/reserved-instances/pricing/