Wigiwork — Remote Tech, AI & Fintech Talent MarketplaceWigiwork — Remote Tech, AI & Fintech Talent Marketplace
Log inSign up
agileyelena's avatar
@agileyelena

Big Data Developer, Enterprise Data Lake Optimization

$14,000/mo· Negotiable

New York City, NY

Experience: 9 yearsOpen to: New York City, NY·

Wigi highlights

9+ years, currently operating as Senior Big Data Developer

Developed Spark code with Scala and Spark-SQL/Streaming

Built a job server and job shell for job submission and monitoring

In the current role, they used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive and implemented solutions for ingesting data from various sources and processing data at rest using Hadoop, MapReduce frameworks, MongoDB, Hive, Oozie, Flume, Sqoop and Talend.

In a previous role, real-time feed was extracted using Kafka and Spark streaming, converted to RDD, processed as Data Frame and saved as Parquet format in HDFS. Also created multi-node Hadoop and Spark clusters in AWS instances to generate terabytes of data and stored it in AWS HDFS.

Available in 2 weeks

Ready to start in 2 weeks. Interview now. Actively looking.

Skills
Tech stack
Apache SparkApache Spark
ScalaScala
Apache HiveApache Hive
Apache KafkaApache Kafka
Oracle DatabaseOracle Database
PythonPython
Tools & workflows
Cloudera
Big Data
Work experience
· 2 roles

Data Platform Engineer

Company hidden · 5-10 years

Data Platform Engineer

Company hidden · 1-3 years

More on this profile

Unlocks when they connect with you.

  • education· 1

Similar available Data Engineers

View all