Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
poppycanvas's avatar
@poppycanvas

Enterprise Data Pipeline Optimization Engineer

$5,600/mo

Edinburgh

Experience: 10+ yearsOpen to: Edinburgh·

Wigi highlights

11+ years, currently Enterprise Data Pipeline Optimization Engineer

Migrated IBM DataStage from 9.1 to 11.7 Version

Optimized BigQuery performance by partitioning, clustering and denormalizing data

In one role, optimized a source data extraction pipeline by implementing partition techniques on primary keys to distribute data evenly across DataStage nodes. They also developed and deployed microservices on a GKE cluster in RTL environments using Jenkins, Harbor and Spinnaker tools to automate application deployment and testing.

Other work included streamlining routine problems by implementing automation scripts for recurring issues and minimizing manual intervention. Critical batches were monitored with vigilance and failures promptly addressed alongside routine maintenance tasks. General capabilities include cloud computing, critical thinking, data mining, data visualization and Google Data Studio.

Open to a signing bonus

For the right offer and opportunity, it will happen.

+ $5,000 signing bonus
Skills
Tech stack
dbtdbt
TerraformTerraform
Microsoft Power BIMicrosoft Power BI
TableauTableau
PythonPython
IBM DataStageIBM DataStage
Tools & workflows
Spinnaker
Cloud Composer
Pub/Sub
Google Data Studio
GKE
Data Mining
Critical Thinking
Business Requirements
Data Visualization
Cloud Computing
Problem Solving
Data Quality
GCP
Work experience
· 4 roles

Data Systems Engineer

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 1-3 years

ETL Systems Engineer

Company hidden · 3-5 years

IT Systems Architect

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • education· 1

Similar available Data Engineers

View all