Inference Infrastructure Engineer

About the role

Rhoda AI is building generalist intelligent robots and needs an Inference Infrastructure Engineer to design and operate the systems that power model deployment across cloud and on‑prem environments.

Key responsibilities

• Design and run large‑scale infrastructure for model workloads on cloud and on‑prem.
• Build and maintain Kubernetes‑based pipelines for distributed ML workloads.
• Own GPU resource scheduling, utilization optimization, and cost‑performance trade‑offs.
• Integrate and manage ML frameworks and serving tools such as Triton, Ray Serve, and TorchServe.
• Develop tooling for model deployment, versioning, and observability.
• Contribute to reliability and scalability as model complexity grows.

Required profile

• 3+ years of experience in ML infrastructure, MLOps, or distributed systems.
• Strong proficiency with Kubernetes and containerized deployment pipelines.
• Experience with GPU orchestration and resource scheduling across large distributed jobs.
• Hands‑on experience with cloud providers (AWS, GCP) and hybrid cloud/on‑prem setups.
• Familiarity with ML frameworks (PyTorch, JAX) and model serving tools (Triton, Ray Serve, TorchServe).

Required skills

• Kubernetes
• GPU orchestration
• AWS
• GCP
• PyTorch
• JAX
• Triton
• Ray Serve
• TorchServe