Principal Data Processing Engineer

About the role

DataPelago is building an innovative data processing engine that modernizes how Apache Spark, Flink, Ray and other platforms handle large‑scale data. As a Principal Data Processing Engineer you will lead the architecture, design and implementation of the core execution components, driving performance, scalability and reliability.

Key responsibilities

• Drive evolution of the parallel and distributed execution engine, focusing on accelerated computing technologies.
• Lead end‑to‑end design, implementation and rollout of enterprise‑grade execution engine components.
• Individually design, implement, test and maintain critical engine modules.
• Analyze academic and industry advances to identify opportunities for differentiation.
• Collaborate with engineering, product management and customer success teams, and mentor junior engineers.

Required profile

• BS/MS in Computer Science or related field with 10+ years of relevant experience.
• 7+ years developing core components of enterprise‑grade database or analytics execution engines for large‑scale data.
• Proven experience leading teams of 10+ engineers on high‑performance data processing engines.
• Strong analytical, problem‑solving and communication skills.

Required skills

• Proficiency in C, C++ and Rust.
• Extensive development experience on Linux.
• Hands‑on experience with Apache Spark, Apache Flink, Apache Doris, Apache Gluten, Velox, Apache DataFusion/Comet.

What we offer

• Technical leadership on a category‑defining product.
• Opportunities to work on cutting‑edge accelerated computing challenges.
• Direct impact on performance and scalability of a mission‑critical platform.
• Mentorship responsibilities and personal growth.
• Competitive compensation, stock options and comprehensive benefits.