Performance Engineer, Inference Systems

About the role

Anthropic is looking for a Performance Engineer to work on its inference fleet that powers Claude for millions of users. The role spans the entire inference stack—from accelerator kernels to routing and autoscaling—ensuring the system delivers high throughput, low latency, reliability, and correct model outputs.

Key responsibilities

• Conduct cross‑layer performance investigations, quantify gaps between actual fleet performance and theoretical rooflines, and identify high‑impact root causes.
• Own and enhance the correctness evaluation pipeline that validates model output quality across hardware, numerics, and serving configurations.
• Develop observability, dashboards, and modeling tools to make performance, cost, reliability, and correctness metrics transparent across the stack.
• Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and implement optimizations.
• Rank and triage a large set of optimization opportunities, focusing on those with the greatest impact.

Required profile

• Hands‑on performance engineering experience in profiling, roofline analysis, latency/throughput optimization, and root‑cause investigation in complex production systems.
• Ability to read, instrument, and contribute to large Python codebases.

Required skills

• Python