Academic Researcher – Remote (30+ hrs/week)

About the role

We are launching a frontier model evaluation initiative to advance next‑generation large language model systems across technical and professional domains. The role involves designing and validating real‑world benchmark tasks, building executable Python problem sets, and evaluating model outputs.

Key responsibilities

• Design and validate challenging, real‑world benchmark tasks rooted in your academic or professional expertise.
• Build executable Python‑based problem sets with clear specifications and verifiable test cases for agentic workflows.
• Evaluate model outputs to identify reasoning, logic, and problem‑solving failures in complex scenarios.
• Develop structured gold‑standard solutions and rubrics to support consistent evaluation frameworks.
• Analyze system behavior to surface capability gaps and failure modes in advanced reasoning tasks.
• Collaborate with domain experts across STEM and quantitative disciplines to improve evaluation quality and rigor.

Required profile

• Current or retired professor, or PhD candidate, in STEM or professional disciplines such as computer science, mathematics, physics, engineering, statistics, economics, finance, law, or related fields.
• Strong academic background from a top‑tier university or equivalent research environment.
• Working proficiency in Python applied in research, coursework, or professional settings.
• Ability to design and implement executable problem‑solving tasks and computational workflows.
• Strong analytical thinking with ability to assess logical correctness and system behavior.
• Ability to work independently for at least 30 hours per week on weekdays.

Required skills

• Python