
Big Data Engineer, Enterprise-Scale Data Processing
Sofia
Wigi highlights
10+ years, most recently Senior Big Data Engineer
Reduced data quality processing from ~36000 core-hours to ~25 core-hours
Designed and maintained 100+ data pipelines for datasets from hundreds of GB to TB
In the most recent role, repartitioned petabyte-scale datasets to eliminate small file inefficiencies and improve query performance, and migrated and operated pipelines across on-prem (HDFS/YARN) and AWS (EMR/S3/Delta Lake).
In a previous role as a QA Engineer, they also implemented monitoring and reporting solutions across a range of environments and platforms and developed statistical analysis of call center and its operators' efficiency. Skills include Apache Airflow, AWS Glue, CI, Distributed Systems and Django.
Available in 2 weeks
Ready to start in 2 weeks. Interview now. Actively looking.
Skills
Work experience
Unlocks when they connect with you.
- certifications· 1
- education· 2
Similar available Data Engineers
View all- Data Engineer (Real-Time Analytics Pipelines) · Białystok, Poland · 10+ Yrs Exp · Remote · $5,100/mo
- Data Engineer (Real-Time Well Analytics) · Nowy Sącz, Poland · 10+ Yrs Exp · On-site · $4,900/mo
- Senior Data Engineer (Data-Driven Campaign Optimization) · Amersfoort, Netherlands · 10+ Yrs Exp · On-site · $5,400/mo
- Data Engineer (Scalable Analytics Pipelines) · Paterson, NJ · 6 Yrs Exp · Remote · $11,300/mo
- Enterprise Data Enablement Engineer · Warsaw, Poland · 6 Yrs Exp · Hybrid · $3,800/mo