
Senior Scalable Data Pipeline Engineer
Paris
Wigi highlights
7+ years, currently operating as Senior Scalable Data Pipeline Engineer
Refactors Spark Scala pipelines to optimise AWS costs
Optimised Azure costs via dynamic HDInsight cluster management
Current responsibilities include integrating new sources into an AWS Data Lake using AWS Glue (Scala) and orchestrating workflows with Apache Airflow (Lambda and Glue operators).
In a previous role, they also optimised Spark Java/PySpark processing on large volumes and developed multi-source PySpark ETL/ELT pipelines for financial instruments and positions. The general skill set includes Amazon S3, Big Data, Data Architecture, Data Engineering and Data Warehousing (DWH).
Available in 2 weeks
Ready to start in 2 weeks. Interview now. Actively looking.
Skills
Work experience
Unlocks when they connect with you.
- certifications· 1
- education· 4
Similar available Data Engineers
View all- Data Engineer (Microsoft Power BI, SQL) · Iași, Romania · 10+ Yrs Exp · Remote · $2,400/mo
- Senior Data Engineer (Real-Time Analytics) · Singapore · 2 Yrs Exp · Remote · $3,900/mo
- Senior Apache Kafka Big Data Engineer | Scala · Odesa, Ukraine · 10+ Yrs Exp · Remote · $2,400/mo
- Data Engineer — Jira & Email Automation · Adelaide, Australia · 10+ Yrs Exp · Remote · $5,900/mo
- Senior Data Engineer / Log Analysis & Petroleum Geology · Cluj-Napoca, Romania · 10+ Yrs Exp · Remote · $3,000/mo