Member of Technical Staff – Data Ingestion Engineer

About the role

Data is becoming a core driver of AI breakthroughs. As a member of the Data Team, you will build and operate large‑scale ingestion systems that turn web and other massive data sources into reliable, well‑structured corpora for training frontier models.

Key responsibilities

• Design, build, and run large‑scale data ingestion pipelines, including web crawling, extraction, normalization, versioning, and delivery to pre‑training pipelines.
• Run experiments to evaluate crawling strategies, extraction methods, and trade‑offs, and use results to improve system performance.
• Analyze ingested data to identify gaps, redundancy, and opportunities for improvement.
• Develop specialized crawlers for high‑priority data sources and ensure pipelines scale reliably across massive campaigns.
• Review code, debug production issues, and continuously enhance ingestion infrastructure.

Required profile

• Curious about how training data influences model capabilities and able to iterate quickly based on measurable impact.
• Strong collaborator who can work closely with researchers, infrastructure, operations, and external partners.
• Enjoys a hybrid research‑engineering role that blends system building with experimental evaluation.

Required skills

• Experience building web crawling, data ingestion, or large‑scale data acquisition systems using Ray, Beam, Spark, or similar technologies.
• Familiarity with LLM training and evaluation pipelines and intuition for useful training data.
• Comfort working with multi‑TB to PB‑scale datasets and building observable, testable, maintainable systems.
• Ability to design experiments and use data‑driven insights to guide system improvements.
• Excellent communication skills for explaining system behavior and trade‑offs.

What we offer

• Top‑tier compensation with salary and equity.
• Stock options for all contributors.
• Comprehensive health, dental, vision, and life coverage plus wellness allowance.
• Daily lunch and dinner provided on‑site.
• 22 weeks paid parental leave for all new parents.
• Unlimited PTO in the U.S. (30 days in the U.K.).
• Visa sponsorship and long‑term immigration support.
• Regular off‑sites, happy hours, and team celebrations.