Engineering Lead, Data Ingestion

About the role

Reflection's Data team builds the training corpora for frontier models. As the Data Ingestion Lead you will guide the team that transforms web and licensed sources into structured, versioned datasets for pre‑training.

Key responsibilities

• Build, mentor and grow a high‑performing team of data ingestion engineers.
• Provide front‑line leadership across web crawling, extraction pipelines and data lakes, ensuring scalability, reliability and auditability.
• Stay hands‑on to contribute to the stack and prioritize day‑to‑day execution of data acquisition campaigns.
• Guide architectural decisions on crawler scheduling, distributed processing (Ray/Beam/Spark), orchestration (Airflow/Prefect) and storage formats (Parquet, JSONL, WARC).
• Partner with pre‑training, data quality and legal teams to align ingestion strategies with model impact and licensing constraints.

Required profile

• Proven experience building and leading data or infrastructure engineering teams while remaining technically hands‑on.
• Deep experience with web‑scale data acquisition or ingestion systems at multi‑TB to PB scale.
• Strong coding ability and credibility to earn technical trust of senior engineers.

Required skills

• Web crawling and acquisition technologies.
• Distributed compute frameworks: Ray, Beam, Spark.
• Orchestration tools: Airflow, Prefect.
• Data formats and storage: Parquet, JSONL, WARC, object‑store / lakehouse architectures.

What we offer

• Top‑tier compensation with salary and equity.
• Comprehensive health, dental, vision and life benefits plus wellness allowance.
• Daily lunch and dinner provided on‑site.
• Unlimited PTO in the U.S. and generous parental leave.
• Visa sponsorship and support for long‑term immigration pathways.