[Feature] Extend EXECUTION_MODE.AIRFLOW_ASYNC for dbt-snowflake adapter

[pankajkoti]

Description

Continuing the work introduced in Cosmos 1.7 to support executing dbt project resources in a deferrable fashion using Airflow provider operators, this feature aims to extend EXECUTION_MODE.AIRFLOW_ASYNC to the dbt-Snowflake adapter.

Initially, this approach was implemented for BigQuery and refined through PRs #1474 and #1483 to ensure robustness. The next step is to explore how we can extend this functionality to Snowflake, leveraging its capabilities to execute dbt models asynchronously while integrating seamlessly with Airflow’s deferrable operators.

Explore how we can utilize existing deferrable operators from the Apache Airflow Snowflake provider or astronomer-providers. If suitable operators are unavailable, consider adding the necessary components to the Cosmos codebase to enable deferrable execution efficiently and accelerate development.

Use case/motivation

This enhancement will improve efficiency, reduce resource consumption, and provide users with a more scalable and optimized way to run dbt projects on Snowflake within Airflow.

Related issues

No response

Are you willing to submit a PR?

• Yes, I am willing to submit a PR!