
Data Engineer, Regulated Event-Scoring And Alerting Systems
London
Wigi highlights
7+ years, currently working as Mid-level Data Engineer
Reduced end-to-end batch runtime by ~30% through parallel-execution tuning
Led migration from Airflow on legacy servers to Dagger on OpenShift
As a trade flow data engineer, built batch ingestion pipelines in Java/Spark and PySpark consuming AVRO market data (T-1) from the Market Data Lake - orders, trades, quotes, XVol - producing partitioned Parquet on HDFS/Hive for the surveillance scenario engines. Engineered FCA/MAR surveillance scenarios - Wash Trading, Front Running, Momentum Ignition - across preprocessing, filtering, model scoring, enrichment; alerts persisted to MDL/HDFS and surfaced to investigators via the Surveillance Workbench (SQL Server).
In a separate role, built an Azure-native data pipeline for a fraud-tolerant phone insurance claims platform, ingesting claim events, scoring them against fraud rules and models and producing alerts for the claims investigation team. Applying trade-surveillance engineering patterns to fraud detection resulted in a faster delivery cycle than a from-scratch build. Skills include Apache Hive, Apache Kafka, Azure Data Lake Storage and Azure DevOps.
Available now
Ready to start immediately. No notice period. Actively looking.
Skills
Work experience
Unlocks when they connect with you.
- certifications· 1
- education· 3
- Files· 6
Similar available Data Engineers
View all- Senior Big Data Developer, Regulated Data Pipelines · Lviv, Ukraine · 9 Yrs Exp · Remote · $2,700/mo
- Data Engineer (Quantitative Risk Modeling) · Washington, DC · 6 Yrs Exp · Hybrid · $11,600/mo
- Data Engineer (Cloud-Native Pipelines) · Minneapolis, MN · 10+ Yrs Exp · Hybrid · $10,900/mo
- Data Engineer | Multi-body Dynamics & Simulink · Norwich, UK · 10+ Yrs Exp · Hybrid · $5,800/mo
- Python & SQL Server Data Engineer · Berlin, Germany · 9 Yrs Exp · Remote · $5,700/mo