Member of Technical Staff – Evaluations

About the role

Join Reflection, a research lab focused on open and accessible AI, as a Member of Technical Staff in Evaluations. You will drive critical comparative analysis and build evaluation systems that create tight feedback loops between data, evaluations, and model behavior.

Key responsibilities

• Conduct critical comparative analysis to advance understanding of model capabilities.
• Build and refine evaluation systems and processes that link data, evals, and model behavior.
• Develop generalizable evaluation frameworks for reasoning, alignment, and usefulness.
• Collaborate with pre‑training, post‑training, and applied teams to translate insights into model improvements.
• Push the boundaries of measurability using synthetic evals, human feedback, and real‑world interaction data.

Required profile

• Strong statistical analysis and experimental design skills for rigorous measurement of model improvements.
• Familiarity with LLM evaluation methodologies such as static benchmarks, human preference evaluations, and agentic tasks.
• High agency, thriving in a fast‑paced startup environment with a bias for impact over process.
• Motivated by building feedback loops that enable models to truly improve.

Required skills

• Statistical analysis
• Experimental design
• LLM evaluation methodologies

What we offer

• Top‑tier compensation: salary and equity.
• Stock options for all contributors.
• Comprehensive health, dental, vision, and life coverage plus annual wellness allowance.
• Daily lunch and dinner provided on‑site.
• 22 weeks paid parental leave for all new parents.
• Unlimited PTO in the U.S. (30 days in the U.K.).
• Visa sponsorship and immigration support.
• Regular off‑sites, happy hours, and team celebrations.