Senior Inference Optimization ML Engineer

About the role

Rhoda AI is building generalist intelligent robots and needs an Inference Optimization ML Engineer to make its foundation models run fast and efficiently in production, both in the cloud and on‑robot.

Key responsibilities

• Own end‑to‑end inference performance, diagnosing and improving latency, throughput, and efficiency of large multimodal models.
• Build systematic performance attribution, identify bottlenecks, and prioritize optimizations across model families.
• Apply and develop techniques such as quantization, pruning, distillation, operator fusion, and model compilation (TensorRT, torch.compile, XLA).
• Optimize attention mechanisms, KV caching, and memory layouts for vision, video, language, and proprioception models.
• Work with kernel‑level tooling (CUDA, Triton) to create or tune custom kernels and build benchmarking infrastructure.

Required profile

• 3+ years of experience in inference optimization, ML systems, or a closely related field.
• Deep hands‑on experience with modern ML stacks (PyTorch required; JAX a plus).
• Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference.
• Exceptional debugging and measurement ability to turn “inference is slow” into clear bottlenecks and validated improvements.

Required skills

• PyTorch, JAX
• CUDA, Triton
• TensorRT, TorchServe, vLLM
• Quantization (INT8/FP8/AWQ), pruning, distillation, model compilation
• Kernel‑level optimization and benchmarking tools