Forward Deployed Engineer - Language Models
Artificial Analysis · San Francisco
Job description
About the role
Artificial Analysis, an independent AI benchmarking company, is hiring a Forward Deployed Engineer to own the day-to-day operation of its language model benchmarking stack. The role acts as the technical face of the company toward major AI labs, onboarding new models, running and debugging benchmarks, and serving as the main technical contact for customers. It is available on-site in San Francisco (5 days a week) or Melbourne (3 days a week).
Key responsibilities
- Operate and maintain the Python-based benchmarking pipeline end-to-end: onboard new models, configure evaluations, run benchmarks and validate results.
- Debug issues across the stack, from API endpoint timeouts to unexpected benchmark outputs.
- Serve as the primary technical contact for AI lab customers, communicating results, explaining methodology and troubleshooting integrations via Slack and video calls.
- Monitor benchmark runs for anomalies and ensure the accuracy of published results.
- Maintain documentation of processes, known issues and model configurations, and collaborate with the engineering team on improvements.
- Stay current with new model releases and API changes.
Required profile
- 3+ years in a client-facing technical role such as solutions engineering, support engineering or technical consulting.
- Strong written and verbal English, able to explain technical concepts clearly to technical stakeholders.
- Highly responsive, reliable and detail-oriented, comfortable with operational, repeatable work.
Required skills
- Strong Python proficiency and comfort with complex codebases you did not write.
- Hands-on experience with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.).
- Excellent debugging skills across APIs, data pipelines and code.
Nice to have: experience with AI evaluation or benchmarking, familiarity with LLM inference infrastructure, background in AI labs or model providers, and B2B SaaS or developer tools experience. Competitive compensation including equity.
Questions fréquentes
Why are you reporting this job?
Explore further
Salaries, guides and searches in the United States.
Salary: Forward Deployed Engineer Based on 54 job offers in the United StatesSalaries by job title
- Delivery Specialist 330
- Line Cook 264
- Restaurant Host 251
- Dishwasher 241
- Shift Supervisor 226
- Parts Specialist 208
- Retail Service Specialist 176
- Server - Red Robin 140
Apply in 30 seconds
Enter your email to apply. An account will be created automatically.
By continuing, you accept our terms of use.
Already have an account? Login
A question about this job?
Ask it here: you will get the full job summary by e-mail, right away.
Published 20 hours ago
Expires 1 month from now
11 views · 0 interested
Boost your chances
Upload your CV — we will match you with relevant openings.
Analyzing your CV...
Artificial Analysis
San Francisco