
Python Data Engineer (Google BigQuery)
Amsterdam
Wigi highlights
11+ years, currently working as Mid-level Python Data Engineer
Implemented CI/CD pipelines with Jenkins for Spark
Optimized Spark jobs processing 200-300 GB of data
Fine-tuned Spark applications to handle complex join operations involving more than 15 sources in a single dataset, significantly improving the performance and efficiency of the data processing pipeline; developed data assets using PySpark to perform transformations and mappings, facilitating loading of data from multiple banking system portfolios into GCS.
Developed Sqoop scripts to import data from Teradata, ensuring data integration accuracy and efficiency; employed a Test-Driven Development approach, resulting in functional and unit test cases with 100% code coverage.
Open to offers
Open to new opportunities
Skills
Work experience
Unlocks when they connect with you.
- certifications· 8
- education· 2
Similar available Data Engineers
View all- Data Integration Engineer (Optimizing Data Pipelines) · Kyiv, Ukraine · 10+ Yrs Exp · On-site · $3,600/mo
- Data Engineer, Data Infrastructure For Marketing Analytics · Chernivtsi, Ukraine · 4 Yrs Exp · Remote · $2,300/mo
- Senior Data Warehouse Specialist (HTML, Microsoft Power BI) · Leiden, Netherlands · 10+ Yrs Exp · Remote · $8,600/mo
- Data Engineer (Python & Big Data) · Poznań, Poland · 4 Yrs Exp · Remote · $3,400/mo
- Data Engineer, Optimizing Data Pipelines For Scalable Insights · Manchester, UK · 10+ Yrs Exp · Remote · $5,100/mo