
Data Engineer, AI-Driven Data Pipeline Optimization
Palo Alto, CA
Wigi highlights
3+ years, currently working as Mid-level Engineer
Improved multi-label classification performance by 42% with DistilBERT
Designed a resume-job pipeline that boosted matching scores by 17%
As a science intern, they enhanced an ETL pipeline using Python-based regex and an NER model, ensuring high-quality data in PostgreSQL, reducing database usage by 28% and improving job recommendations. In another role, built a personnel risk-level classification model using LightGBM in Python, handling highly imbalanced data with SMOTE and Tomek Links and achieving an AUC exceeding 0.8.
An OOD detection approach using the CLIP vision-language model with positive-and-negative prompt learning in PyTorch was also engineered, optimizing class-specific prompts on an ImageNet-100 dataset to achieve 97.79% AUC. They also customized genetic algorithm modules for FJSP using Geatpy in Python, cutting computation time by 45% to 82% compared to an MIP solver and delivering high-quality results for medium-to-large scale problems (up to 1000 jobs) in industrial applications.
Available now
Ready to start immediately. No notice period. Actively looking.
Skills
Work experience
Unlocks when they connect with you.
- projects· 6
- education· 2
Similar available Data Engineers
View all- Qlik Data Engineer | Jenkins · Barcelona, Spain · 10+ Yrs Exp · On-site · $5,100/mo
- Senior Data Engineer / Data Pipelines & PySpark · Utrecht, Netherlands · 10+ Yrs Exp · Remote · $7,600/mo
- Junior Big Data Developer — Scrum & Maven · Málaga, Spain · 2 Yrs Exp · Remote · $3,900/mo
- Data Engineer, Biodiversity Data Infrastructure · Amsterdam, Netherlands · 6 Yrs Exp · Remote · $6,100/mo
- Data Engineer (Solutions De Données Massives) · Paris, France · 6 Yrs Exp · Remote · $7,300/mo