Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
emeraldkurapika's avatar
@emeraldkurapika

Senior Data Engineer (Real-Time Analytics Infrastructure)

$4,500/mo

Kraków

Experience: 10+ yearsOpen to: Kraków·

Wigi highlights

12+ years, most recently Senior Data Engineer

Migrated 600+ jobs to a new Ab Initio version

Collected data from 75+ sources and processed 120-150 GB hourly

In an earlier role, collected CDC data from 56 tables from different sources and processed them using a Data Bricks pipeline to find business insight. They also developed a data ingestion and transformation workflow in Data Bricks using PySpark and Oracle.

Separate work involved implementing the main pipeline for partitioning to UTC and computing data on Databricks using AWS, PySpark and Snowflake. A high-level Data Quality solution was also designed and implemented on top of the main pipeline to validate different metrics.

Available in 2 weeks

Ready to start in 2 weeks. Interview now. Actively looking.

Skills
Tech stack
Apache AirflowApache Airflow
Oracle DatabaseOracle Database
Apache HiveApache Hive
SQLSQL
Apache SparkApache Spark
JenkinsJenkins
Microsoft ExcelMicrosoft Excel
PythonPython
AWSAWS
DatabricksDatabricks
Tools & workflows
Control Center
Data Modeling
Data Architects
Statistical Data Analysis
GitHub
Data Analysis
Ab Initio
Spotfire
Autosys
PySpark
Work experience
· 5 roles

Senior Big Data Engineer

Company hidden · 1-3 years

Senior Big Data Engineer

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 1-3 years

Big Data ETL Specialist

Company hidden · 3-5 years

BI & ETL Systems Engineer

Company hidden · 3-5 years

More on this profile

Unlocks when they connect with you.

  • education· 1
  • Photos· 4

Similar available Data Engineers

View all