Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
undinedamselfly's avatar
@undinedamselfly

Data Engineer, Optimizing Retail Data Workflows

$5,700/mo

Cambridge

Experience: 5 yearsOpen to: Cambridge·

Wigi highlights

5+ years, currently working as Mid-level Data Engineer

Used complex SQL queries to help reduce costs by 15%

Built a star-schema database enabling sub-millisecond data analysis

Created a Python class which cleans and transforms over 120k rows of data before being loaded into a Postgres database; developed a system that extracts retail sales data from five different data sources: PDF documents, an AWS RDS database, a RESTful API, JSON and CSV files.

Developed an end-to-end data processing pipeline hosted on AWS based on Pinterest's experimental processing pipeline, using Lambda architecture for both batch and stream-processing; streamed data in real-time using AWS Kinesis and performed near real-time analysis leveraging a Spark cluster on Databricks.

Open to offers

Open to new opportunities

Skills
Tech stack
Apache SparkApache Spark
Microsoft SQL ServerMicrosoft SQL Server
MySQLMySQL
FastAPIFastAPI
Tools & workflows
Problem Solving
Business Requirements
Data Cleaning
Written Communication
CI
Continuous Delivery (CD)
Query Writing
Data Warehousing (DWH)
Microsoft Office
Data Manipulation
ETL Pipeline Design
Data Pipelines
Data Architecture
Work experience
· 3 roles

Data Systems Engineer

Company hidden · 3-5 years

Cybersecurity Engineer

Company hidden · 1-3 years

Enterprise Systems Analyst

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • projects· 4
  • certifications· 6
  • education· 5

Similar available Data Engineers

View all