
Senior Data Engineer / Data Pipelines & PySpark
Utrecht
Wigi highlights
12+ years, currently operating as Senior Engineer
Implemented end-to-end Azure pipelines with Data Factory and Synapse
Built GCP-based ETL/ELT batch and real-time Kafka streaming pipelines
As a senior big engineer, developed a real-time streaming pipeline using Cloud Run Dataflow that provided insights into customer behavior with a latency of under 5 seconds, and developed PySpark applications on Databricks, which yielded a significant 30% amount of cost reduction.
In a separate role as engineering lead, they also built GCP Cloud Run REST APIs using services such as BigQuery, GCS, Pub/Sub, Airflow, Cloud Workflows and Cloud Scheduler and used Azure DevOps, Google Cloud Build, triggers, Jenkins and Airflow for scheduling and programming.
Available in 2 weeks
Ready to start in 2 weeks. Interview now. Actively looking.
Skills
Work experience
Unlocks when they connect with you.
- certifications· 11
- education· 1
Similar available Data Engineers
View all- Sr Data Engineer (Optimizing Pipelines Across Cloud Platforms) · Porto, Portugal · 5 Yrs Exp · Remote · $3,300/mo
- Data Engineer (Automated Analytics Infrastructure) · Lutsk, Ukraine · 4 Yrs Exp · Remote · $2,500/mo
- Data Engineer & Analyst · Leuven, Belgium · 10+ Yrs Exp · On-site · $5,700/mo
- ETL Developer | Talend & ETL · Warsaw, Poland · 5 Yrs Exp · Remote · $3,600/mo
- Data Engineer (Optimizing Reporting Pipelines) · Bordeaux, France · 6 Yrs Exp · Remote · $6,200/mo