
Senior Data Engineer | DataQuality Engineer
Singapore
Wigi highlights
7+ years, currently working as Senior Data Engineer
Designed ETL for millions of daily counterparty records
Built ETL workflows with Spark, Airflow, Python, S3 and Hive
As part of the current responsibilities, partitioning and compression techniques were implemented to optimize data storage, enhancing query performance and reducing costs for large volumes of historical data. In a previous role, they also used Apache Spark, HDFS, S3, Airflow, Hive and Athena to build scalable, efficient workflows.
Other work included designing and implementing ETL pipelines to enable statistical insights on large datasets, comparing on-prem and cloud-based data pipelines. Also built and optimized data pipelines to process student activity from RDBMS and S3, ensuring accurate and timely academic performance reporting. Their general capabilities include AWS Glue, AWS Lambda, Big Data Analytics and Data Streaming.
Open to offers
Open to new opportunities
Skills
Work experience
Unlocks when they connect with you.
- certifications· 6
- education· 1
Similar available Data Engineers
View all- Junior Big Data Developer — Scrum & Maven · Málaga, Spain · 2 Yrs Exp · Remote · $3,900/mo
- Data Engineer, Biodiversity Data Infrastructure · Amsterdam, Netherlands · 6 Yrs Exp · Remote · $6,100/mo
- Data Engineer (Solutions De Données Massives) · Paris, France · 6 Yrs Exp · Remote · $7,300/mo
- Data Engineer & Tech Lead · Katowice, Poland · 10+ Yrs Exp · Remote · $3,900/mo
- Data Engineer (Scalable Pipeline Engineering) · Kyiv, Ukraine · 4 Yrs Exp · Remote · $2,800/mo