Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
darkice4's avatar
@darkice4

Python & AWS Lambda Data Engineer

$9,500/mo

Grand Rapids, MI

Experience: 5 yearsOpen to: Grand Rapids, MI

Wigi highlights

4+ years, currently working as Mid-level Python & AWS Lambda Data Engineer

Cut report execution time from 120 seconds to 3 seconds

Built PySpark validation frameworks with over 80 rules

In the current role, processes over 400 GB of data daily across multiple source systems, ensuring high availability, reliability and data consistency. During a previous tenure, they also migrated over 30 legacy ETL workflows from on-premises environments to Azure cloud platforms, reducing infrastructure and maintenance costs.

Other work included developing scalable Spark workloads for large telecom datasets and optimizing performance using partitioning, broadcast joins, caching and Delta Lake techniques. Also automated source-to-target reconciliation processes and implemented data quality validation frameworks, reducing manual effort while improving data accuracy.

Available now

Ready to start immediately. No notice period. Actively looking.

Skills
Tech stack
SnowflakeSnowflake
Azure Data FactoryAzure Data Factory
Apache AirflowApache Airflow
AWS LambdaAWS Lambda
GitGit
Amazon RedshiftAmazon Redshift
Amazon S3Amazon S3
AWS GlueAWS Glue
Apache KafkaApache Kafka
AWSAWS
Microsoft AzureMicrosoft Azure
PythonPython
SQLSQL
Tools & workflows
Aure Databrick
Sypark
Delta Lake
Data Warehousing (DWH)
ETL
PySpark
Azure Databricks
Work experience
· 3 roles

Data Systems Engineer

Company hidden · 1-3 years

Azure ETL Engineer

Company hidden · 1-3 years

Data Engineering Associate

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • certifications· 9
  • education· 2

Similar available Data Engineers

View all