Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
vegapoet's avatar
@vegapoet

Senior Databricks Data Engineer (Data Architecture)

$13,400/mo

New York City, NY

Experience: 10+ yearsOpen to: New York City, NY·

Wigi highlights

12+ years, currently operating as Senior Data Engineer

Migrated MapReduce jobs to Spark, improving processing speed

Led on-premises data infrastructure migration to cloud platforms

As a senior big data engineer, they design and manage end-to-end Databricks infrastructure, including cluster optimization, job orchestration and performance tuning, and build, optimize and maintain scalable data pipelines for batch and streaming workloads across multiple domains.

In an earlier role, also implemented real-time data streaming using Kafka, enabling timely insights and actionability, and leveraged Azure Cosmos DB for globally distributed, low-latency data storage and ensured seamless data access. The general capabilities include AWS, Azure Data Factory and Data Analytics.

Available in 2 weeks

Ready to start in 2 weeks. Interview now. Actively looking.

Skills
Tech stack
Microsoft AzureMicrosoft Azure
Azure Data FactoryAzure Data Factory
Apache HadoopApache Hadoop
Apache SparkApache Spark
PythonPython
ScalaScala
Microsoft SQL ServerMicrosoft SQL Server
Apache KafkaApache Kafka
AWSAWS
LinuxLinux
MySQLMySQL
SQLSQL
Tools & workflows
Data Architecture
Data Maintenance
Data Architects
Azure Databricks
Big Data
Data Processing
Data Analytics
Data Modeling
Work experience
· 7 roles

Senior Big Data Engineer

Company hidden · 1-3 years

Data Systems Engineer

Company hidden · 3-5 years

Data Systems Engineer

Company hidden · under 1 year

Data Pipeline Engineer

Company hidden · 1-3 years

Hadoop Platform Engineer

Company hidden · 1-3 years

Data Pipeline Engineer

Company hidden · 1-3 years

Java Software Engineer

Company hidden · under 1 year

More on this profile

Unlocks when they connect with you.

  • certifications· 3
  • education· 1
  • Files· 1

Similar available Data Engineers

View all