chore: Moving function examples to dedicated folder (#2)

example_functions/README.md

@@ -0,0 +1,7 @@
+# Example functions
+
+Here you can find examples on how to leverage Jinja templating with the CLI to perform dynamic imports.
+
+# How to use them
+Each example should include the full importable content (all depdencies included) so that you can just trigger an import pointing to the example folder. 
+Some examples might have additional dependencies that must be installed, so make sure to check the example's `README` file.

example_functions/fetch_db_credentials_from_aws/README.md

@@ -0,0 +1,15 @@
+# Fetching DB credentials from AWS Secrets Manager Store
+
+This example covers how to dynamically fetch DB credentials from the AWS Secrets Manager Store. This is an example using an Athena DB connection, but can be altered to different DB engines as needed.
+
+# How to use it
+1. Make sure to install the additional dependencies listed in the `requirements.txt` file.
+2. Replace in the `functions/database.py` file (lines 5 ~ 9):
+    1. `<AWS_SECRET_ID>` with the ID for the desired secret from AWS Secrets Manager.
+    2. `<ATHENA_REGION_NAME>` with the AWS region for the Athena instance.
+    3. `<ATHENA_SCHEMA_NAME>` with the schema from Athena you're connecting to.
+    4. `<S3_BUCKET_NAME>` with the S3 bucket name for the staging directory (where query results are stored).
+    5. `<ATHENA_WORK_GROUP>` with the Athena Work Group that should be used for the connection.
+3. Trigger a CLI import command pointing to this directory.
+
+You'll also need to set up authentication for the `boto3` package (the AWS SDK for Python). Please refer to [this article](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html).

databases/Athena.yaml → example_functions/fetch_db_credentials_from_aws/databases/Athena.yaml

@@ -1,5 +1,5 @@
 database_name: Athena
-sqlalchemy_uri: "{{ functions.database.get_sqlalchemy_uri(filepath) }}"
+sqlalchemy_uri: "{{ functions.database.get_sqlalchemy_uri() }}"
 cache_timeout: null
 expose_in_sqllab: true
 allow_run_async: false

example_functions/fetch_db_credentials_from_aws/functions/database.py

@@ -0,0 +1,40 @@
+from functools import lru_cache
+import boto3
+import json
+
+AWS_SECRET_ID = "<AWS_SECRET_ID>"
+ATHENA_REGION_NAME = "<ATHENA_REGION_NAME>"
+ATHENA_SCHEMA_NAME = "<ATHENA_SCHEMA_NAME>"
+S3_BUCKET_NAME = "<S3_BUCKET_NAME>"
+ATHENA_WORK_GROUP = "<ATHENA_WORK_GROUP>"
+
+@lru_cache()
+def _get_db_secrets() -> str:
+  """
+  Get secrets from AWS Secrets Manager.
+  """
+  try:
+    secrets_client = boto3.client('secretsmanager')
+    response = secrets_client.get_secret_value(
+      SecretId=AWS_SECRET_ID
+    )
+  except Exception as e:
+    raise Exception(f"Failed to fetch secrets from AWS Secrets Manager: {e}")
+
+  return json.loads(response['SecretString'])
+
+
+def get_sqlalchemy_uri() -> str:
+  """
+  Returns the SQLAlchemy URI for an Athena DB connection.
+  """
+
+  aws_secrets_data = _get_db_secrets()
+  athena_iam_user_access_key_id = aws_secrets_data["athena-iam-user-access-key-id"]
+  athena_iam_user_secret_key = aws_secrets_data["athena-iam-user-secret-key"]
+
+  return (
+    f"awsathena+pandas://{athena_iam_user_access_key_id}:{athena_iam_user_secret_key}"
+    f"@athena.{ATHENA_REGION_NAME}.amazonaws.com/{ATHENA_SCHEMA_NAME}"
+    f"?s3_staging_dir=s3://{S3_BUCKET_NAME}/&work_group={ATHENA_WORK_GROUP}"
+  )

example_functions/fetch_db_credentials_from_aws/requirements.txt

@@ -0,0 +1 @@
+boto3==1.35.72