Member of Technical Staff – Model Efficiency

About the role

Cohere is building cutting‑edge foundation AI models and needs engineers to boost inference efficiency. As a Member of Technical Staff on the Model Efficiency team, you will improve latency, throughput and quality of large language model deployments.

Key responsibilities

• Analyze the end‑to‑end inference stack, identify performance bottlenecks and implement optimizations.
• Collaborate with modeling and systems teams to experiment, measure impact and ship production improvements.
• Develop GPU/CUDA kernels, low‑level system tweaks, and execution strategies for MoE and other large‑scale architectures.
• Build expertise in advanced performance techniques such as speculative decoding and KV‑cache optimizations.

Required profile

• 5+ years of experience writing high‑performance, production‑quality code.
• Strong programming skills in C++ or Python (Rust/Go welcome).
• Hands‑on experience with large language models and the LLM inference ecosystem (e.g., vLLM, SGLang).
• Proven ability to diagnose and resolve performance bottlenecks across the model execution stack.
• Bias for action – ship fast, measure impact, iterate quickly.

Required skills

• C++
• Python
• Rust or Go
• GPU programming and CUDA
• Low‑level systems optimization
• Transformers, MoE, speculative decoding, KV‑cache optimizations
• Distributed systems scaling
• vLLM, SGLang

What we offer

• Weekly lunch stipend of $75.
• Full health, dental and mental‑health benefits.
• RRSP matching, 401(k) and pension scheme.
• 100% parental‑leave top‑up for up to 6 months.
• Annual enrichment budget for arts, culture, fitness and workspace improvements.
• Education stipend for conferences, courses and coaching.
• 6 weeks paid vacation plus travel budget for office visits.
• $500 home‑office setup stipend and co‑working benefit.