Member of Technical Staff – Data Ingestion & Quality

About the role

We are looking for a Member of Technical Staff to own the end‑to‑end lifecycle of data sets that fuel our large‑scale physics‑based AI models. The role focuses on discovering, ingesting, and guaranteeing the quality of multimodal physical data, ensuring it is ready for training at petabyte scale.

Key responsibilities

• Research and acquire new modalities of physical data (sparse sensors, point clouds, hyperspectral imagery, radar) through partnerships, vendors, and public archives.
• Design, build, and operate petabyte‑scale data pipelines (e.g., Apache Spark) for batch and streaming ingestion, including orchestration, storage, and monitoring.
• Develop and maintain quality metrics that assess coverage, correctness, and consistency across data sources.
• Create automated QA checks that continuously monitor data quality and generate actionable verdicts.
• Write technical requirements and provide feedback to external data vendors and partners.
• Collaborate with researchers to validate that improved data sets translate into better model performance.

Required profile

• Proven experience building large‑scale data pipelines, QA systems, or evaluation workflows.
• Strong attention to detail for detecting subtle data inconsistencies that affect model performance.
• Ability to quickly understand unfamiliar data formats, sensor documentation, and vendor specifications.
• Experience working with external data vendors and partners.

Required skills

• Apache Spark
• Ray
• Beam