Overview
With the Snowflake announcement of supporting dbt core natively in their new workspaces, it seems to have opened the door to easily adopt dbt directly in the cloud and in the data warehouse itself. This makes it easier to onboard new team members and even upskill existing team members to start using dbt directly in snowflake.
This article will not go over the details to set up dbt in snowflake, but rather discuss the tradeoffs of using dbt in snowflake, dbt cloud and dbt core in a self-hosted environment. Snowflake also made another announcement with the release of dbt fusion, which is slightly different to dbt core. Lets forgot about fusion for now and carry on with core only discussion.
dbt core DX in Snowflake
Here some of the resources I found useful while setting up dbt core in snowflake:
I think its nice that snowflake workspaces now support dbt core, but its not a full IDE yet and lacks a lot of features that you would get from a local development environment.
Positives
DAG view is nice and really responsive, clicking on different model opens up the sql/python script in the workspace instantly (not almost instantly, its is instantly). Couple that with the side view that shows what kind of model it is and the location of the model is nifty feature.
Adding a drop down the pick the run the profile is a great DX experience and
Table view of tradeoffs
| Feature/Aspect | dbt CLI (SAND) | dbt Cloud | dbt in Snowflake |
|---|---|---|---|
| Ease of Setup | Moderate | Easy | Easy |
| Cost | Low | Subscription Fee | Included in Snowflake Costs |
| Performance | High | Moderate | High |
| Collaboration | Limited | Excellent | Good |
| Version Control | Manual | Built-in | Manual |
| CI/CD Integration | Manual | Built-in | Manual |
| Learning Curve | Moderate | Low | Low |
| Community Support | Strong | Strong | Growing |
dbt cloud alternative
Not even close, dbt lab should be thanking snowflake for this integration as it will drive more users to dbt cloud. dbt cloud is a fully managed service that provides a lot of features that dbt core does not provide, such as:
- Built-in CI/CD
- Built-in version control
- Built-in collaboration tools
- Built-in documentation