Assignment

Chose the Data Engineering Question for my job application - ML Engineer. This repository contains all the code and documentation. My attempt landed me a ML Engineer role which I did not accept.

Deliverables:

1. Documentation.md:
   • Design Documentation (Includes introduction, system architecture, assumptions);
   • Setup Documentation on steps to set up the system;
   • Maintenance for common problems;
2. results.csv
   • Example of a daily weather result CSV file that is stored in an AWS S3 bucket.

Prerequisites

For this walkthrough, you should have the following prerequisites:

• AWS Account;
• AWS CLI;
• AWS SAM CLI;
• Conda/Miniconda.

References:

https://aws.amazon.com/blogs/devops/micro-etl-pipeline-how-to-fetch-process-and-refresh-small-data-using-aws-lambda-and-aws-sam/

License

MIT