
AWS Lambda & Apache Spark Data Engineer
Allen, TX
Wigi highlights
7+ years, currently Senior AWS Lambda & Apache Spark Data Engineer
Developed Delta Lake and Apache Iceberg workflows with Spark/PySpark
Implemented ETL workflows to migrate data to OneStream
In the current role, implemented modern Lakehouse data management practices, including incremental processing, MERGE/upsert operations, partitioning, schema evolution, snapshots and time travel to support scalable and reliable data processing. During a previous tenure, they also built Databricks and Delta Lake pipelines using PySpark for real-time data ingestion, cleansing, validation and transformation, while monitoring and optimizing Spark workloads.
Separate work included implementing Delta Lake Bronze/Silver/Gold pipelines with incremental ingestion, MERGE/upserts, schema management, deduplication, null handling and source-to-target reconciliation. Also developed data migration workflows from Amazon S3 to Snowflake using Scala and Spark to support efficient and reliable data processing. General capabilities include Amazon Kinesis, Amazon QuickSight and Amazon RDS.
Open to offers
Open to new opportunities
Skills
Work experience
Unlocks when they connect with you.
- education· 1
Similar available Data Engineers
View all- Data Engineer (Solutions De Données Massives) · Paris, France · 6 Yrs Exp · Remote · $7,300/mo
- Data Engineer & Tech Lead · Katowice, Poland · 10+ Yrs Exp · Remote · $3,900/mo
- Data Engineer (Scalable Pipeline Engineering) · Kyiv, Ukraine · 4 Yrs Exp · Remote · $2,800/mo
- Enterprise Data Integration Engineer · Hamburg, Germany · 8 Yrs Exp · Hybrid · $7,000/mo
- Data Engineer, Enterprise-Scale Data Processing Pipelines · Jena, Germany · 4 Yrs Exp · Remote · $5,500/mo