Member of Technical Staff, ML Platform

About the role

We are seeking an ML infrastructure engineer to own model evaluation at Runway, turning scattered evaluation work into a unified platform. You will design systems that generate samples at scale, score them with automated metrics and human annotations, and deliver results to researchers quickly.

Key responsibilities

• Own the end‑to‑end evaluation platform, including tooling for generation, annotation, review and adaptation at frontier scale.
• Define and implement CLIs, APIs, GUIs and storage layers to make the evaluation process seamless and collaborative.
• Work directly with research teams on video, image, audio, agents and robotics to understand measurement needs and generalize solutions across the platform.
• Set standards for reproducibility, metric definitions, reporting formats, and trustworthiness of results.
• Support broad adoption and integration of the platform across training, production serving and other research efforts.

Required profile

• 5+ years building ML infrastructure or data platforms in production, including evaluation or benchmarking systems.
• Strong Python and PyTorch expertise, with hands‑on experience running large batch GPU workloads on Kubernetes.
• Experience designing data pipelines and storage for large media or model outputs, focusing on versioning and reproducibility.
• Familiarity with experimental statistics such as paired comparisons, confidence intervals and inter‑rater agreement.
• Ability to lead a technical area, gather requirements, set direction and drive a roadmap independently.

Required skills

• Python
• PyTorch
• Kubernetes
• GPU batch processing
• Data pipeline design
• Experimental statistics