Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
lukeglide's avatar
@lukeglide

Data Engineer

$5,500/mo

Argenteuil

Experience: 10+ yearsOpen to: Argenteuil·

Wigi highlights

10+ years, most recently Junior Data Engineer

Designed a star schema with SCD2 historization in 21 dbt models

Developed anomaly detection with Spark Structured Streaming

In a project, they deployed the pipeline on AWS with Terraform: MSK Serverless, EMR Serverless with IAM authentication, and persistent S3 storage, Glue and Athena, separate from the ephemeral infrastructure to control costs. Separate work involved writing transformations with dbt and orchestrating the pipeline with Airflow on Amazon MWAA, in a private network.

Other work included calculating five business KPIs (OTD, PPM, lead time, Days of Supply, Supplier Health Score) and orchestrating the pipeline with Airflow. An aberrant indicator (915 days of stock) was corrected to a coherent value (3.2 days) after fixing three root causes identified by SQL queries.

Open to offers

Open to new opportunities

Skills
Tech stack
PostgreSQLPostgreSQL
Apache AirflowApache Airflow
TerraformTerraform
dbtdbt
PythonPython
SQLSQL
AWSAWS
Microsoft Power BIMicrosoft Power BI
Microsoft AzureMicrosoft Azure
DockerDocker
KubernetesKubernetes
Apache KafkaApache Kafka
SnowflakeSnowflake
MongoDBMongoDB
Tools & workflows
Data Modelling
Streaming
Apache Spark / PySpark
Data Pipelines / ETL
NoSQL
Work experience
· 2 roles

Procurement Specialist - Anti-Vibration Components

Company hidden · 3-5 years

Head of Electric Vehicle/Hybrid Machine Development

Company hidden · 5-10 years

More on this profile

Unlocks when they connect with you.

  • projects· 3
  • certifications· 7
  • education· 5

Similar available Data Engineers

View all