Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
cosmictag's avatar
@cosmictag

Data Engineer, Enterprise-Scale Data Pipeline Architect

$11,400/mo

Chicago, IL

Experience: 10+ yearsOpen to: Chicago, IL·

Wigi highlights

12+ years, currently working as Senior Data Engineer

Migrated 40+ Talend ETL jobs to Databricks + Delta Lake

Engineered CDC + SCD Type 1/2 on Delta Lake across 15+ critical datasets

In the current role, built a real-time ingestion framework using Event Hubs, Kafka and Stream Analytics, processing 500M+ daily financial events at sub-minute latency. They also architected a Bronze→Silver→Gold lakehouse on Delta Lake with partitioning, Z-ordering and vacuum policies, reducing storage costs 35% and cutting query times by half.

During a previous tenure, also architected an ADLS Gen2 + Delta Lake + Auto Loader ingestion layer, eliminating data loss incidents and enabling point-in-time recovery. Kafka + Flink + Spark Streaming event pipelines were built as well, enabling real-time analytics on 200M+ daily healthcare operational events.

Open to a signing bonus

For the right offer and opportunity, it will happen.

+ $5,000 signing bonus
Skills
Tech stack
SnowflakeSnowflake
Azure Data FactoryAzure Data Factory
Tools & workflows
Palantir
Snowflake SQL
Snowflake Time Travel
Snowpipe
Virtual Warehouses
Clustering Keys
Snowflake Stream & Tasks
Snowflake Data Sharing
Snowflake Stages
Databricks SQL Analytics
Unity Catalog
Delta Tables
Optimization with Z-Ordering
Autoloader Schema Evolution
Cluster Management
Azure Databricks
PySpark
PL/SQL
Work experience
· 4 roles

Senior Cloud Data Engineer (Azure)

Company hidden · 3-5 years

Azure ETL Engineer

Company hidden · 3-5 years

Data Platform Engineer

Company hidden · 1-3 years

Data Warehouse ETL Developer

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • certifications· 3
  • education· 2
  • Files· 5

Similar available Data Engineers

View all