Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
derefkeeper's avatar
@derefkeeper

Sr. Scala Data Engineer | Apache Kafka

$11,800/mo

Raleigh, NC

Experience: 10+ yearsOpen to: Raleigh, NC·

Wigi highlights

12+ years, currently operating as Sr. Scala Data Engineer

Built an ETL framework with Talend-Big Data and Kafka for CDC ingestion

Automated Big Query provisioning and management with Terraform

In a previous role, developed an ingestion framework using Python to ingest data from LIS files into Snowflake, replacing a legacy application, and implemented Azure Data Factory pipelines to parse XML data and store it securely, ensuring data integrity, availability and seamless integration with telecommunications applications.

Separate work included developing a Spark streaming pipeline to read data from Kafka and write it to HDFS and a Spark framework using Scala to load data from various RDBMS and streaming sources into Hive.

Available now

Ready to start immediately. No notice period. Actively looking.

Skills
Tech stack
Apache HadoopApache Hadoop
PythonPython
SnowflakeSnowflake
JavaJava
Apache SparkApache Spark
ScalaScala
Apache OozieApache Oozie
Apache KafkaApache Kafka
Apache PigApache Pig
SQLSQL
Tools & workflows
Sqoop
HiveQL
NoSQL
Apache Spark Streaming
Azkaban
PL/SQL
Cloudera
Algorithms
Data Structures
Big Data Analytics
Work experience
· 7 roles

Senior Data Engineer

Company hidden · 5-10 years

Senior Data Engineer

Company hidden · under 1 year

Senior Big Data Developer

Company hidden · under 1 year

Senior Big Data Architect

Company hidden · under 1 year

Data Systems Engineer

Company hidden · under 1 year

Senior Code Engineer

Company hidden · 1-3 years

Software Engineer

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • certifications· 4
  • education· 1

Similar available Data Engineers

View all