Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
slickcapella3's avatar
@slickcapella3

Apache Spark Data Integration Engineer Skilled in Redshift

$8,600/mo

Delaware

Experience: 4 yearsOpen to: Delaware

Wigi highlights

4+ years, currently working as Mid-level Data Integration Engineer

Cut daily ETL execution time by 60% with PySpark on EMR

Built Airflow and AWS Glue pipelines, cutting batch failures by 45%

Created modular dbt models for fact tables, reducing SQL duplication by 70% and enabling faster updates to metric logic across multiple stakeholder-facing dashboards; configured CloudWatch alarms and integrated SNS alerts, reducing incident response time from 90 to 50 minutes for critical overnight job failures.

In a previous role, automated ETL from MySQL to Snowflake using Python and Cron, improving daily refresh reliability to 98% and removing the need for manual updates; managed 40+ Airflow DAGs to ingest JSON and CSV, reducing end-to-end data load time by over 3 hours across environments.

Open to offers

Open to new opportunities

Skills
Tech stack
CronCron
MySQLMySQL
PandasPandas
SQLSQL
DockerDocker
KubernetesKubernetes
Apache KafkaApache Kafka
Microsoft AzureMicrosoft Azure
ScalaScala
Apache SparkApache Spark
Tools & workflows
ETL
JSON
CSV
Redshift
PySpark
Database Systems
Data Structures
Object-Oriented Programming (OOP)
Artificial Intelligence (AI)
Work experience
· 2 roles

Data Systems Engineer

Company hidden · 1-3 years

Data Engineering Associate

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • education· 2
  • Photos· 2

Similar available Data Engineers

View all