Staff Software Engineer, Inference Infrastructure

About the role

Cohere is looking for a Staff Software Engineer to join its Model Serving team and build high‑performance, scalable, and reliable machine‑learning infrastructure. You will help define and build the next generation of AI platforms that deliver large language models through easy‑to‑use API endpoints.

Key responsibilities

• Develop, deploy, and operate the AI platform that serves Cohere’s large language models with low latency, high throughput, and high availability.
• Collaborate with multiple product and engineering teams to optimize NLP model deployments in production environments.
• Work directly with customers to create customized deployments that meet specific performance and reliability requirements.

Required profile

• 5+ years of engineering experience running production infrastructure at large scale.
• Proven experience designing highly available distributed systems with Kubernetes and GPU workloads.
• Hands‑on experience with multi‑cloud environments (GCP, Azure, AWS, OCI) and hybrid/on‑prem deployments.
• Strong troubleshooting skills in complex Linux‑based computing environments and a grit to solve evolving technical challenges.

Required skills

• Kubernetes
• GPU workloads
• GCP, Azure, AWS, OCI
• Linux
• Golang or C++
• Distributed systems

What we offer

• Weekly lunch stipend of $75 (or equivalent in local currency).
• Full health, dental, and mental‑health benefits.
• RRSP matching, 401(k), and pension scheme.
• 100% parental‑leave top‑up for up to six months.
• Annual enrichment budget for arts, culture, fitness, and workspace improvements.
• Education stipend for conferences, courses, and coaching.
• Six weeks of paid vacation and travel budget for office visits.
• Remote‑friendly work model with co‑working benefits and a $500 home‑office stipend.