Migration· Case study 05
Warehouse & lakehouse to ClickHouse, with reverse ETL and integrity checks
Migration paths that move analytics data from Snowflake and lakehouse tables into ClickHouse and Apache Druid, and back out through reverse ETL, with automated checks that confirm the numbers match at every hop.
Every hop
reconciled for drift, hourly
Architecture at a glance
- Snowflake / lakehouse
- S3 / GCS
- ClickHouse
- Reverse ETL
01 The problem
Customers keep their source of truth in Snowflake or lakehouse formats but need fast, high-concurrency analytics that warehouses are too slow or too expensive to serve.
02 What I built
- 1A staged path, warehouse or lakehouse → object storage (S3/GCS) → ClickHouse, with partitioned, incremental, idempotent loads.
- 2Support for Parquet, Apache Iceberg, and Delta Lake sources, with ClickHouse schemas designed for the query patterns (sort keys, partitions, LowCardinality, codecs).
- 3Reverse ETL that exports modeled and aggregated results from ClickHouse or Druid back to warehouses, object storage, and downstream tools.
- 4A sidecar integrity-check project that compares key metrics across Snowflake, object storage, and ClickHouse over recent hourly windows, and alerts on drift.
- 5Cutover and rollback plans agreed with customer data teams.
03 Impact
Warehouse-grade data became real-time analytics at a fraction of the query cost, and every stage of the pipeline is continuously checked for drift.