Posts

Background Its almost given that for any Python project I have build in the past 2 year I have been using uv as a way to manage dependencies and virtual environments. Using uv has drastically improved my development experience and as an added bonus improved my development speed. That being I was tired of running the same command over and over again: source .venv/bin/acttivate This is not a big problem but a problem none the less, for each project, multiple times a day I was manually running this command, either auto filling the terminal shell or looking it the past history and running it. On multiple occasional I forgot to run it and it broke the magical zoned-it-zone. ...

TLDR Every corporationg have their own network securities polcies, they use network proxies to control and monitor the traffic going in and out of their network. Just assume that the proxy will be different corporation. When working with Snowflake Privatelink, you will have to get the proxies to whitelist the Snowflake private endpoints. This will only provide you access to Snowflake when interacting directly with Snowflake. When using IaC tools like Terraform, you will have to configure the proxy settings in your local environment so that Terraform can access Snowflake endpoints via the corporate proxy. This is the simple part, as you just need to declare the HTTP_PROXY and HTTPS_PROXY environment variables. ...

Overview By adding the tags auto-propagation feature in Snowflake, it effectively shifted the data governance to the left at the source level and removed the burden and responsibility of identifying downstream columns which are potentially breaking the data governance strategy and exposing sensitive data due to an human error. In Snowflake, best practices for data governance is to use masking policies associated with tags. For sensitive data, masking at column level is the most common and best practice, so tags are applied at column level in Snowflake. ...

Overview With the Snowflake announcement of supporting dbt core natively in their new workspaces, it seems to have opened the door to easily adopt dbt directly in the cloud and in the data warehouse itself. This makes it easier to onboard new team members and even upskill existing team members to start using dbt directly in snowflake. This article will not go over the details to set up dbt in snowflake, but rather discuss the tradeoffs of using dbt in snowflake, dbt cloud and dbt core in a self-hosted environment. Snowflake also made another announcement with the release of dbt fusion, which is slightly different to dbt core. Lets forgot about fusion for now and carry on with core only discussion. ...