Staff / Senior Software Engineer – Inference

About the role

Anthropic is building reliable, interpretable, and steerable AI systems. The Inference team powers Claude, serving millions of users through large‑scale, compute‑agnostic deployments. This role designs and operates the distributed infrastructure that delivers high‑performance inference across diverse AI accelerators.

Key responsibilities

• Design, build, and maintain distributed systems that serve Claude globally.
• Develop resilient, real‑time adaptive services for production workloads.
• Implement intelligent request routing, load balancing, and traffic management across thousands of accelerators.
• Maximize compute efficiency via autoscaling and orchestration of production, research, and experimental jobs.
• Create and operate production‑grade deployment pipelines for new model releases.
• Provide high‑performance inference infrastructure to enable next‑generation model research.
• Integrate emerging AI accelerator platforms and support new model architectures.

Required profile

• Significant software engineering experience, especially with distributed systems.
• Results‑oriented mindset with a bias toward flexibility and impact.
• Willingness to take on tasks beyond the core job description.
• Interest in learning about machine‑learning systems and infrastructure.
• Ability to thrive where technical excellence drives both business outcomes and research breakthroughs.
• Awareness of the societal impact of AI work.

Required skills

• Distributed systems
• High‑performance large‑scale systems
• Machine‑learning systems at scale
• Load balancing
• Request routing
• Traffic management
• LLM inference optimization, batching, and caching strategies
• Kubernetes
• Autoscaling and orchestration
• Production deployment pipelines