Member of Technical Staff – Training Infrastructure

About the role

Join causal’s mission to build a Large Physics foundation Model that can predict and control physical systems. As a Member of Technical Staff you will make large‑scale training fast, efficient, and reliable, enabling every GPU cycle to accelerate research progress.

Key responsibilities

• Design, implement, and optimise distributed training systems that scale across thousands of GPUs.
• Research and test parallelisation strategies, numerical‑precision trade‑offs, and memory‑optimisation techniques for novel model architectures.
• Analyse, profile, and debug low‑level GPU operations to maximise throughput and hardware utilisation.
• Build reusable frameworks for checkpointing, fault tolerance, and reproducibility that remain robust under rapid research iteration.
• Collaborate with researchers to bring prototype architectures to full‑scale production.
• Stay up‑to‑date with the latest research and incorporate new ideas into the training stack.

Required profile

• Relentless problem‑solver with rapid execution and a strong ability to learn in unfamiliar domains.
• Proven experience with distributed training frameworks such as FSDP, DeepSpeed, Megatron, PyTorch, or JAX/XLA.
• Deep understanding of state‑of‑the‑art optimisation techniques: parallelism strategies, memory optimisation, mixed precision, and communication overlap.
• Ability to profile and debug performance from framework internals down to kernels and collectives.
• Strong grasp of deep‑learning frameworks (e.g., PyTorch, JAX) and their system architectures.
• Bonus: contributions to open‑source ML infrastructure projects.

Required skills

• PyTorch
• JAX / XLA
• DeepSpeed
• Megatron
• FSDP
• GPU profiling and debugging (e.g., Nsight, CUDA tools)
• CUDA programming