Member of Technical Staff - Pre-Training Infra

About the role

Join Reflection's research lab to build and scale distributed training systems that power frontier model pre‑training. Work closely with research teams to design, operate, and optimize large‑scale training runs for foundation models.

Key responsibilities

• Develop infrastructure enabling efficient training across thousands of GPUs using modern distributed frameworks.
• Optimize training throughput, stability, and efficiency for large model workloads.
• Collaborate with pre‑training researchers to translate experimental ideas into production‑ready systems.
• Build and maintain training pipelines supporting large datasets, checkpointing, and experiment iteration.
• Debug and resolve performance bottlenecks across distributed stacks, including model parallelism and GPU communication.

Required profile

• Experience building or operating distributed training systems for large ML models.
• Strong background working with modern distributed training frameworks such as Megatron or DeepSpeed.
• Familiarity with large‑scale model parallelism strategies (data, tensor, pipeline, or expert parallelism).
• Proven ability to optimize GPU utilization and training throughput in distributed environments.
• Experience collaborating with ML researchers to productionize experimental workflows.

Required skills

• Megatron
• DeepSpeed
• NCCL (GPU communication library)
• Model parallelism techniques
• Large‑scale ML pipeline development

What we offer

• Top‑tier compensation with salary and equity.
• Stock options for all contributors.
• Comprehensive health, dental, vision, and life coverage plus wellness allowance.
• Daily lunch and dinner provided on‑site.
• 22 weeks paid parental leave for all new parents.
• Unlimited PTO in the U.S. and 30 days in the U.K.
• Visa sponsorship and immigration support.
• Regular team‑building events and off‑sites.