Researcher – Evaluations Lead

About the role

The New Horizons Evaluations team is reimagining how we measure progress in interactive machine intelligence. As Evaluations Lead, you will design evaluation frameworks that capture not just what models know — but how they reason, remember, and interact over time. You’ll work at the intersection of research, product, and infrastructure to develop metrics, systems, and studies that define what “intelligence” means in the next generation of AI.

Key responsibilities

• Identify and define key model capabilities and behaviors that matter for next‑generation model evals.
• Develop and implement new evaluation pipelines with robust statistical analysis and clear reporting.
• Partner closely with model training and research teams to embed evaluation systems directly into model development loops.
• Prototype new user studies and behavioral experiments to ground evaluations in real‑world use.

Required profile

• Experience designing or implementing evaluation frameworks for generative models (audio, text, or multimodal).
• Strong technical and analytical skills, ability to take open‑ended research ideas and translate them into production‑ready systems.
• Creativity in defining novel quantitative metrics for subjective or behavioral qualities.
• Excitement for building evaluation systems that bridge research and real‑world use.
• Curiosity and rigor in equal measure; motivation driven by discovering how to measure meaningful progress in intelligent behavior.

What we offer

• Competitive base salary alongside attractive equity package.
• Fully covered medical insurance along with dental and vision for you and your family.
• 9 weeks paternity & 12 weeks maternity leave.
• 401(k) retirement plan.
• Monthly commuter allowance stipend.
• Flexible PTO to recharge.
• Daily lunch, dinner and plenty of snacks provided.