Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
royalregister's avatar
@royalregister

Senior Scalable Data Pipeline Engineer

$7,300/mo

Paris

Experience: 8 yearsOpen to: Paris·

Wigi highlights

7+ years, currently operating as Senior Scalable Data Pipeline Engineer

Refactors Spark Scala pipelines to optimise AWS costs

Optimised Azure costs via dynamic HDInsight cluster management

Current responsibilities include integrating new sources into an AWS Data Lake using AWS Glue (Scala) and orchestrating workflows with Apache Airflow (Lambda and Glue operators).

In a previous role, they also optimised Spark Java/PySpark processing on large volumes and developed multi-source PySpark ETL/ELT pipelines for financial instruments and positions. The general skill set includes Amazon S3, Big Data, Data Architecture, Data Engineering and Data Warehousing (DWH).

Available in 2 weeks

Ready to start in 2 weeks. Interview now. Actively looking.

Skills
Tech stack
AWSAWS
AWS GlueAWS Glue
ScalaScala
Apache SparkApache Spark
Apache AirflowApache Airflow
JavaJava
Microsoft AzureMicrosoft Azure
Amazon S3Amazon S3
Tools & workflows
Data Warehousing (DWH)
Data Architects
Data Architecture
Data Engineering
PySpark
ELT
ETL
Pipeline Development
Big Data
GDPR
Work experience
· 5 roles

Senior Big Data Engineer

Company hidden · 1-3 years

Senior Big Data Engineer

Company hidden · 3-5 years

Functional Consultant

Company hidden · under 1 year

Django Developer

Company hidden · under 1 year

Web Developer

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • certifications· 1
  • education· 4

Similar available Data Engineers

View all