Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
heimerruntime's avatar
@heimerruntime

Apache Airflow & PySpark Data Engineer

$10,000/mo

Stafford, VA

Experience: 5 yearsOpen to: Stafford, VA·

Wigi highlights

4+ years, currently operating as Mid-level Data Engineer

Developed Spark applications using Python and Apache Spark

Created Spark ETL pipelines on AWS EMR for various data sources

Designed and implemented data pipelines utilizing AWS Redshift stored procedures to facilitate seamless data movement and transformation; implemented data validation and cleansing procedures using SQL to maintain high data quality standards and accuracy throughout the data processing pipeline.

In a previous role, they also developed ETL/ELT pipelines using big data and cloud technologies such as Apache Hive, Apache Spark, Azure Data Factory and Azure Databricks and extracted, transformed and loaded data from source systems to Azure Data Storage services using Azure Data Factory, T-SQL, Spark-SQL and U-SQL Azure Data Lake Analytics.

Open to offers

Open to new opportunities

Skills
Tech stack
Apache HiveApache Hive
Apache SparkApache Spark
Apache AirflowApache Airflow
Amazon RedshiftAmazon Redshift
Amazon EC2Amazon EC2
PythonPython
SQLSQL
AWSAWS
Tools & workflows
PySpark
Databases
S3
AWS Cloud
GitHub
REST APIs
Test-Driven Development (TDD)
Data Verification
Teradata SQL
Version Control
Data Integration
ETL
Work experience
· 3 roles

Data Systems Engineer

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 1-3 years

Data Engineering Trainee

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • education· 2
  • Files· 1

Similar available Data Engineers

View all
    Apache Airflow & PySpark Data Engineer (Hybrid) | Wigiwork