Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
speedykingdom's avatar
@speedykingdom

Data Engineer | Apache Spark & Data Governance

$5,600/mo

Regensburg

Experience: 6 yearsOpen to: Regensburg

Wigi highlights

4+ years, most recently Senior Data Engineer

Resolved ingestion bottlenecks, improving performance by 30%

Orchestrated daily operations for a 3-person support team

In a past position, maintained and monitored 100+ tables and 180 interfaces with master and transactional data from SAP and enterprise systems, ensuring data integrity across the full lifecycle. They also developed robust ADF pipelines integrated with Azure API Management (APIM) and Azure Service Bus Topics, ensuring reliable data ingestion and event-driven messaging.

Separate work involved architecting scalable cloud-native lakehouses on the Databricks Data Intelligence Platform using Delta Live Tables (DLT), utilizing Unity Catalog to enforce automated data governance and lineage across production layers. Also developed parameterised ADF pipelines to ingest data from Azure SQL Database, ADLS (CSV, JSON, Parquet) and Data Storage Tables, with support for incremental loading and historical backfilling.

Available now

Ready to start immediately. No notice period. Actively looking.

Skills
Tech stack
Apache SparkApache Spark
PythonPython
Microsoft AzureMicrosoft Azure
Azure Data FactoryAzure Data Factory
DatabricksDatabricks
MySQLMySQL
SQLSQL
Tools & workflows
Data Quality
Data Governance
Event-Driven
Data Lakes
Cost Efficiency
GitHub
Data AI
Data Modeling
ETL
Analytics
PySpark
Data Domain
Big Data
Work experience
· 6 roles

Data Systems Engineer

Company hidden · under 1 year

Senior Business Consultant - Risk Mitigation

Company hidden · under 1 year

Senior Technical Lead

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 3-5 years

Technical Project Lead

Company hidden · 1-3 years

Web Development Trainee

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • certifications· 12
  • education· 3

Similar available Data Engineers

View all