
Senior Data Engineer, Cloud Data Systems Optimization
Philadelphia, PA
Wigi highlights
11+ years, currently operating as Senior Data Engineer
Led large-scale data migration to Azure using Snowflake
Implemented real-time data processing with Kafka and Spark Streaming
As part of the current role, designed and implemented a self-hosted integration runtime in Azure Data Factory, facilitating seamless data transfer from multiple sources such as SQL databases, Oracle databases, CSV files and REST APIs into Azure Blob Storage. They also developed interactive Power BI dashboards, delivering real-time insights into business metrics and enhancing decision-making across teams.
In a previous role, also spearheaded CI/CD processes in Azure DevOps, automating ETL pipeline deployments and ensuring version control and collaboration across teams. Advanced optimization techniques in Databricks Spark, such as partitioning, caching and compression, were implemented to boost performance and scalability of big data workflows.
Open to a signing bonus
For the right offer and opportunity, it will happen.
Skills
Work experience
Unlocks when they connect with you.
- education· 1
Similar available Data Engineers
View all- Data Engineer / Machine Learning & Data Visualization · Budapest, Hungary · 4 Yrs Exp · Remote · $2,300/mo
- Senior Data Engineer - NoSQL & Data Modeling · Parramatta, Australia · 10+ Yrs Exp · Hybrid · $7,000/mo
- Data Engineer, Cloud Data Platform Optimization · Kyiv, Ukraine · 4 Yrs Exp · Remote · $2,800/mo
- Senior Python Data Engineer / Cloud · Eindhoven, Netherlands · 10+ Yrs Exp · On-site · $5,800/mo
- Geodata Engineer / FME Server & ArcGIS Enterprise · Leeuwarden, Netherlands · 10+ Yrs Exp · Hybrid · $5,300/mo