Jobiglo

No results.

Forward Deployed Engineer - Language Models

Artificial Analysis · San Francisco

New
Senior 🇬🇧 English
Python APIs

Job description

About the role

Artificial Analysis, an independent AI benchmarking company, is hiring a Forward Deployed Engineer to own the day-to-day operation of its language model benchmarking stack. The role acts as the technical face of the company toward major AI labs, onboarding new models, running and debugging benchmarks, and serving as the main technical contact for customers. It is available on-site in San Francisco (5 days a week) or Melbourne (3 days a week).

Key responsibilities

  • Operate and maintain the Python-based benchmarking pipeline end-to-end: onboard new models, configure evaluations, run benchmarks and validate results.
  • Debug issues across the stack, from API endpoint timeouts to unexpected benchmark outputs.
  • Serve as the primary technical contact for AI lab customers, communicating results, explaining methodology and troubleshooting integrations via Slack and video calls.
  • Monitor benchmark runs for anomalies and ensure the accuracy of published results.
  • Maintain documentation of processes, known issues and model configurations, and collaborate with the engineering team on improvements.
  • Stay current with new model releases and API changes.

Required profile

  • 3+ years in a client-facing technical role such as solutions engineering, support engineering or technical consulting.
  • Strong written and verbal English, able to explain technical concepts clearly to technical stakeholders.
  • Highly responsive, reliable and detail-oriented, comfortable with operational, repeatable work.

Required skills

  • Strong Python proficiency and comfort with complex codebases you did not write.
  • Hands-on experience with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.).
  • Excellent debugging skills across APIs, data pipelines and code.

Nice to have: experience with AI evaluation or benchmarking, familiarity with LLM inference infrastructure, background in AI labs or model providers, and B2B SaaS or developer tools experience. Competitive compensation including equity.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec Artificial Analysis.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

Why are you reporting this job?

Thank you for your report. We will review this job.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 18 hours ago

Expires 1 month from now

9 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

Artificial Analysis

San Francisco