Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
thyonecoot29's avatar
@thyonecoot29

Data Integration Engineer / Azure Data Factory & Apache Hadoop

$11,600/mo

New York City, NY

Experience: 7 yearsOpen to: New York City, NY·

Wigi highlights

6+ years, currently working as Senior Data Integration Engineer

Optimized PySpark data workflows with efficient joins and caching

Set up Azure DevOps and Jenkins CI/CD pipelines for data solutions

In the current role, designed and implemented data pipelines for banking transaction data, integrating real-time data streaming via AWS Kinesis for faster data processing, and built a data warehouse using AWS Redshift to enable scalable and efficient analytics and reporting for banking data.

In a past role, they also implemented Slowly Changing Dimensions (SCD) Type 1 and Type 2 in Azure Synapse Analytics, enabling accurate historical tracking and reliable reporting. Separate work involved designing data storage solutions using Hadoop HDFS for storing raw network data and making it available for downstream analytics. Amazon DynamoDB is among the listed skills.

Open to offers

Open to new opportunities

Skills
Tech stack
Amazon DynamoDBAmazon DynamoDB
MongoDBMongoDB
Azure Data FactoryAzure Data Factory
Amazon EC2Amazon EC2
Azure DevOpsAzure DevOps
Apache HadoopApache Hadoop
Tools & workflows
NoSQL
Azure Data Lake
Work experience
· 3 roles

Senior Data Engineer

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • certifications· 1
  • education· 1

Similar available Data Engineers

View all