End-to-end Azure data engineering pipeline ingesting real-time earthquake data from the USGS API. Implements a Bronze–Silver–Gold lakehouse using Azure Data Factory, Databricks, ADLS Gen2, and Synapse Analytics, with both manual execution and fully automated daily-triggered workflows.

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

Healthcare CDC validation, Snowflake migration QA, source-to-target reconciliation, and data quality frameworks

This repository is dedicated to showcasing projects built on Databricks, focusing on big data analytics, data engineering, and machine learning workflows.

Panoply is a managed cloud data warehouse and ELT (Extract, Load, Transform) platform that centralizes data from 100+ sources into a BigQuery- or Redshift-backed warehouse. It pairs pre-built SaaS connectors and a customizable Flex Connector with a SQL workbench, a code-free Query Builder, and in-platform dashboards, so startups and data teams get…

Hands-on project demonstrating how to load and query semi/unstructured data in Snowflake. Covered: Loading JSON data into a VARIANT column and extracting structured values using FLATTEN, Loading Parquet files from AWS S3 into relational tables with proper column mapping.

Batch ingestion and ELT pipeline on GCP using Airflow, BigQuery, and dbt

IBM Netezza Performance Server is a cloud-native data warehouse and analytics appliance for running large-scale SQL analytics and in-database machine learning on structured data. Originally a standalone data-warehouse appliance vendor acquired by IBM, Netezza is now delivered as Netezza Performance Server on IBM Cloud, AWS, and Microsoft Azure and…

Wherobots is a managed cloud platform for geospatial analytics and spatial AI at planetary scale, built by the creators of Apache Sedona. WherobotsDB is a distributed Spatial SQL engine with 300+ spatial functions, raster and vector processing (RasterFlow), Havasu (Apache Iceberg-compatible) tables, and data federation across AWS Glue, Databricks…

Automated Data Loading in Snowflake using Snowpipe (End-to-End Implementation).

This project implements a complete end-to-end data engineering pipeline for Airbnb data using modern cloud technologies. The solution demonstrates best practices in data warehousing, transformation, and analytics using Snowflake, dbt (Data Build Tool), and AWS.

❄️🔨End-to-end data engineering project built in Snowflake using a Medallion Architecture (🟫 Bronze → 🟦 Silver → 🟨 Gold). The project demonstrates ELT pipeline design, data ingestion from AWS S3, data cleaning and transformation, incremental processing, and dimensional modelling using a star schema.