This job is no longer available
This job expired on 23/10/2026. It no longer accepts applications.
Member of Technical Staff – Model Evaluations
Simile · San Francisco
Job description
About the role
As a Member of Technical Staff focusing on Model Evaluations, you will design and implement measurement systems that verify the accuracy, trustworthiness, and usefulness of Simile’s human‑behavior simulations. You will help define evaluation metrics, set quality standards, and ensure that evaluation evidence informs model, product, and customer decisions.
Key responsibilities
- Build and maintain robust evaluation pipelines for large‑language‑model (LLM) and behavioral simulations.
- Define quality bars and measurement criteria that align with product and research goals.
- Analyze rich human‑behavior data sources such as surveys, interviews, customer studies, and transactional signals.
- Collaborate with cross‑functional teams (research, engineering, product) to translate evaluation findings into actionable improvements.
- Document evaluation methodology, statistical analyses, and results for internal and external stakeholders.
Required profile
- Strong background in AI/ML model evaluation, research engineering, or statistical analysis.
- Experience working with human‑behavior data, survey methodology, or behavioral science research.
- Ability to reason about distributions, uncertainty, and qualitative outputs.
- Proven track record of turning evaluation evidence into product or model decisions.
Required skills
What we offer
- Opportunity to shape cutting‑edge evaluation frameworks for a fast‑growing AI startup.
- Access to unique, large‑scale human behavior datasets.
- Collaborative environment with top researchers and engineers.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United States.
Salaries by job title
- Delivery Specialist 330
- Line Cook 264
- Restaurant Host 251
- Dishwasher 242
- Shift Supervisor 226
- Parts Specialist 208
- Retail Service Specialist 176
- Server - Red Robin 140
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Simile
San Francisco
Related job offers
-
ML / AI Software Engineer - C++ Metrics Frameworks
General Motors San Francisco -
Frontier Agents Engineer (Forward Deployed Engineering)
Scale AI San Francisco -
Senior Frontier Agents Engineer (Forward Deployed Engineering)
Scale AI San Francisco -
IT Security Analyst
A-LIGN Panama -
Staff Software Developer - Geospatial Applications
Langan Engineering & Environmental Services Panama