Skip to content

Intend of this project is to join data from Orders and Customers tables from Snowlake and generate aggregated data using DBT Core and create new View in SnowFlake DBerate

Notifications You must be signed in to change notification settings

bijuthottathil/Snowflake-DBT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 

Repository files navigation

Data Transformation using DBT Core in SNOWFLAKE

image

Pre requisites Installation of Dbt Core in your machine -

Steps to follow

Create a new folder in your machine then, git clone https://github.com/dbt-labs/dbt-core.git cd dbt-core python -m pip install -r requirements.txt

Make sure that appropriate python and DBT extensions are installed in VS Code

image

image

change the folder and execute

image

After DBT installation, we need to create Virtual Environment python -m venv dbt_venv .\dbt_venv\Scripts\Activate.ps1 ( please make sure that you have powershell installed in your machine)

image

After activation, please go to virtual environment directory

image

Then install snowflake related components for DBT pip install dbt-snowflake

image

After installation, we will focus SnowFlake DB and tables we are going to use

We will be using below highlighted tables

image

Next go to command prompt and start setting up configurations for SnowFlake in DBT

need to run -- DBT Init command

it will prompt to put input values

(dbt_venv) PS D:\DBTSNOWFLAKE\dbt-core\dbt_venv> dbt init 19:22:25 Running with dbt=1.8.6 Enter a name for your project (letters, digits, underscore): snowflakedbt 19:22:49
Your new dbt project "snowflakedbt" was created! whatever snowflake configuration I have, it is provided

image

Whatever entries I added, will be available in profile.yml

image

image

go back to prompt and move to project folder we created

While opening folder you will see below structure

image

Remove all example models generated by DBT

Then add 3 sql model files in the folder

image

image

image

Time to run DBT Run command using

Dbt Run

You can see the result here. Models are generated successfully. You can see additional folders created in vs code

image

Now check the result in SnowFlake

You can see 3 views created

image

Customer_orders is the view , we were expecting with aggregated data

image

You can visualize the details in Snowlake like this

image

Thanks for visiting this page. If you need help to setup this in your machine, please reachout to me

About

Intend of this project is to join data from Orders and Customers tables from Snowlake and generate aggregated data using DBT Core and create new View in SnowFlake DBerate

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published