Cheapest way to run scheduled tasks

[sercantor]

Hi 👋 I have a question @kichik

I have a case where I have a scheduled action that runs every ~10 minutes - think of like a healthcheck for the system, would having a runner on EC2 spot instance be cheaper than having a CodeBuild runner? I do not need the runners to be super fast, and I don't need them to be available at all times

[kichik]

Sounds like a perfect use case for Lambda runners. If you set the ephemeral storage really low (defaults to max 10GB), it might even get on the free tier. If Lambda won't work, then spot instance is probably a cheaper than CodeBuild or even Fargate spot.

[sercantor]

Well I think I failed to mention some details. The job pulls an image from ECR and runs it, and the average runtime is about 1.5-2 minutes, I assume I won't be able to run a docker image in Lambda, if I didn't use a docker image, I'd have to download dependencies everytime I spin up this lambda runner, so it would be around 2.5 minutes I assume.

For this scenario, lambda runner still holds true? I did not expect that, I did try the imagebuilder and did not find it well. I was thinking Fargate spot or ECS runner 😄

[kichik]

If you need to pull and run a Docker image, you would need Docker. That won't work on Lambda and it won't work on Fargate either. Your options would be CodeBuild, ECS or EC2. If you're already using ECS for other jobs, then that would be the best option as it will allow you to share the instances. If not, EC2 should be cheaper both in absolute terms, and also by allowing you to pick instances that are smaller than the smallest CodeBuild compute size. If you can make it work with t4g.nano, for example, that would be the cheapest.

For ECS you can also share a cluster between two providers. One provider would be your main provider with beefy CPU/RAM config. The second provider will be for this job only and use less CPU/RAM so it can fit more easily.

Here is an estimate that might help (based on my original response). For EC2 it assumes the instance is always on so maybe cut that number in half. For Fargate, it doesn't take spot instances into account.

[sercantor]

This is great information. From what you've told me, EC2 seems to be the best option for me right now, as we do not use ECS, and I believet4g.nano can handle the workload.

Thank you so much for answering and even providing calculations @kichik, I'm greatful!