Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
soulfulvulkan's avatar
@soulfulvulkan

Senior Data Engineer (Scalable Cloud Data Pipelines)

$4,300/mo

Athens

Experience: 10+ yearsOpen to: Athens·

Wigi highlights

17+ years, most recently Senior Data Engineer

Implemented a Kafka → Spark Structured Streaming → PostgreSQL pipeline

Built a Docker-based data engineering platform with Spark and Kafka

As a senior big data engineer, they designed and implemented the DataBridge framework, a reusable Airflow-based platform for configurable data extraction, transformation and export workflows. Developed automated SFTP and Amazon S3 ingestion pipelines, performing file validation, data transformations and cloud-based delivery processes.

In an earlier role, also developed automation tools and scripts to reduce manual operational effort and improve operational efficiency. Other work included developing automation scripts and process improvements to increase team efficiency. General capabilities include Apache Hadoop, Azure Databricks, Delta Lake and ETL.

Available in 2 weeks

Ready to start in 2 weeks. Interview now. Actively looking.

Skills
Tech stack
SQLSQL
PythonPython
Apache AirflowApache Airflow
PostgreSQLPostgreSQL
GitLabGitLab
Apache HiveApache Hive
Apache HadoopApache Hadoop
Amazon S3Amazon S3
dbtdbt
Apache KafkaApache Kafka
DockerDocker
LinuxLinux
Tools & workflows
ETL
Data Pipelines
Data Integration
Unix Shell Scripting
Query Writing
PySpark
Azure Databricks
Delta Lake
Work experience
· 6 roles

Senior Big Data Engineer

Company hidden · 5-10 years

IT Systems Engineer

Company hidden · 1-3 years

IT Operations Service Engineer

Company hidden · 3-5 years

Support Systems Engineer

Company hidden · 3-5 years

Software Developer

Company hidden · 1-3 years

Hands-On Learning Lab

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • projects· 1
  • certifications· 1
  • education· 3

Similar available Data Engineers

View all