
Senior Data Engineer (Real-Time Analytics Infrastructure)
Kraków
Wigi highlights
12+ years, most recently Senior Data Engineer
Migrated 600+ jobs to a new Ab Initio version
Collected data from 75+ sources and processed 120-150 GB hourly
In an earlier role, collected CDC data from 56 tables from different sources and processed them using a Data Bricks pipeline to find business insight. They also developed a data ingestion and transformation workflow in Data Bricks using PySpark and Oracle.
Separate work involved implementing the main pipeline for partitioning to UTC and computing data on Databricks using AWS, PySpark and Snowflake. A high-level Data Quality solution was also designed and implemented on top of the main pipeline to validate different metrics.
Available in 2 weeks
Ready to start in 2 weeks. Interview now. Actively looking.
Skills
Work experience
Unlocks when they connect with you.
- education· 1
- Photos· 4
Similar available Data Engineers
View all- Data Engineer (Real-Time Analytics Pipelines) · Białystok, Poland · 10+ Yrs Exp · Remote · $5,100/mo
- Data Engineer (Real-Time Well Analytics) · Nowy Sącz, Poland · 10+ Yrs Exp · On-site · $4,900/mo
- Senior Data Engineer (Data-Driven Campaign Optimization) · Amersfoort, Netherlands · 10+ Yrs Exp · On-site · $5,400/mo
- Data Engineer (Scalable Analytics Pipelines) · Paterson, NJ · 6 Yrs Exp · Remote · $11,300/mo
- Enterprise Data Enablement Engineer · Warsaw, Poland · 6 Yrs Exp · Hybrid · $3,800/mo