Overview

Public list of topics that I want to research and write about in the future. This can be on anything related to data, engineering, software, AI and personal projects(non tech related).

In Works

  • Snowflake and dbt core integration

Development Life cycle

  • Linting and Formatting, is it actually worth doing?
  • gitflow and drawbacks of using it in dataflow
  • Data environments: Does the traditional [dev/uat/prod] environments fit the data products need.

Data Engineering - Pipeline

  • Real-time streaming pipeline with kafka
  • AWS Glue extractiong from an RDS PostGres Server
    • With CDC
    • Without CDC
  • OpenStack Data Lakehouse

Tech Review

  • dbt
  • dlthub
  • SQLMesh
  • github actions
  • Orchestration - Airflow
  • Clickhouse
  • Databend
  • Polars
  • Moose
  • PeerDB
  • Daft
  • RisingWave

Software Development

  • Python Debugger
  • uv: Python Dependency
  • docker
  • terraform

Snowflake

  • RBAC
  • Streamlit development management
  • Snowflake LLM-ETL Integration

Local Development

  • How I use nixOS in my MacBook
  • Local Setup

GenAI

  • Cline
  • CrewAI
  • Agno