Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
speedyfractal's avatar
@speedyfractal

Data Engineer, Regulated Event-Scoring And Alerting Systems

$5,800/mo

London

Experience: 7 yearsOpen to: London·

Wigi highlights

7+ years, currently working as Mid-level Data Engineer

Reduced end-to-end batch runtime by ~30% through parallel-execution tuning

Led migration from Airflow on legacy servers to Dagger on OpenShift

As a trade flow data engineer, built batch ingestion pipelines in Java/Spark and PySpark consuming AVRO market data (T-1) from the Market Data Lake - orders, trades, quotes, XVol - producing partitioned Parquet on HDFS/Hive for the surveillance scenario engines. Engineered FCA/MAR surveillance scenarios - Wash Trading, Front Running, Momentum Ignition - across preprocessing, filtering, model scoring, enrichment; alerts persisted to MDL/HDFS and surfaced to investigators via the Surveillance Workbench (SQL Server).

In a separate role, built an Azure-native data pipeline for a fraud-tolerant phone insurance claims platform, ingesting claim events, scoring them against fraud rules and models and producing alerts for the claims investigation team. Applying trade-surveillance engineering patterns to fraud detection resulted in a faster delivery cycle than a from-scratch build. Skills include Apache Hive, Apache Kafka, Azure Data Lake Storage and Azure DevOps.

Available now

Ready to start immediately. No notice period. Actively looking.

Skills
Tech stack
PythonPython
SQLSQL
Apache SparkApache Spark
JavaJava
Apache KafkaApache Kafka
Apache HadoopApache Hadoop
Apache HiveApache Hive
ParquetParquet
Azure Data FactoryAzure Data Factory
Azure DevOpsAzure DevOps
Microsoft AzureMicrosoft Azure
OpenShiftOpenShift
KubernetesKubernetes
TableauTableau
Microsoft ExcelMicrosoft Excel
GitGit
Tools & workflows
PySpark
HDFS
Avro
Azure Data Lake Storage
Work experience
· 6 roles

Professional Development Coordinator

Company hidden · 1-3 years

Fraud Analytics Data Engineer (Azure)

Company hidden · under 1 year

Trade Flow Data Engineer

Company hidden · 1-3 years

Python Backend Engineer

Company hidden · under 1 year

Python Backend Engineer

Company hidden · 1-3 years

Systems Design Engineer

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • certifications· 1
  • education· 3
  • Files· 6

Similar available Data Engineers

View all