Audio Inference Engineer, Model Efficiency

About the role

Cohere is looking for an Audio Inference Engineer to drive model serving efficiency for audio workloads. You will work on reducing latency, increasing throughput, and maintaining quality for real‑time and streaming audio inference across our enterprise AI platform.

Key responsibilities

• Advance core audio model serving metrics such as latency, throughput, and quality by profiling and optimizing the inference pipeline.
• Identify system bottlenecks and deliver creative, high‑performance solutions for audio processing and streaming workloads.
• Collaborate closely with training and serving infrastructure teams to ensure seamless integration between model development and deployment.
• Focus on real‑time and duplex streaming audio inference, ensuring reliable performance at scale.

Required profile

• Significant experience developing high‑performance audio or machine‑learning inference systems.
• Proficiency with C++ and Python.
• Hands‑on experience with deep‑learning models for audio, speech, or language applications.
• Bias for action and a strong results‑oriented mindset.

Required skills

• C++
• Python
• GPU programming and low‑level system optimization
• Model parallelization techniques over multiple GPUs
• Duplex real‑time streaming architectures
• PyTorch, TensorFlow, or specialized audio libraries
• Inference frameworks such as vLLM, SGLang, Tensort‑LLM, or custom distributed inference systems
• Transformers for audio/speech and end‑to‑end audio pipeline optimization

What we offer

• Weekly lunch stipend of $75 (or equivalent in local currency).
• Full health, dental and mental‑health benefits.
• RRSP matching, 401(k), and pension scheme.
• 100% parental leave top‑up for up to 6 months.
• 6 weeks of paid vacation (30 working days).
• Education and learning stipend for conferences, courses, and coaching.
• Home office stipend of $500 and co‑working benefit for remote employees.
• Travel budget for office visits and an annual company offsite.