Inference Engineer

About the role

We are seeking an Inference Engineer to design and operate the inference layer of our AI cloud platform, Forge. The role focuses on enabling customers to run models without managing GPUs while providing our partners a full‑stack token‑factory solution.

Key responsibilities

• Deploy and serve machine‑learning models on Forge and Kubernetes clusters across global, heterogeneous hardware.
• Evaluate and integrate inference frameworks and serving engines, ensuring production‑ready monitoring, gateways, and endpoints.
• Optimize inference performance through autoscaling, KV‑cache orchestration, speculative decoding, and latency reduction.
• Debug customer inference issues and collaborate with partners to extend token‑factory capabilities.
• Own the end‑to‑end inference stack, from initial deployment to continuous improvement.

Required profile

• Strong general background in AI inference with ability to reason across the full request‑to‑token pipeline.
• Deep hands‑on experience operating production Kubernetes clusters.
• Understanding of inference performance concepts such as TTFT, disaggregated inference, speculative decoding, and KV‑cache mechanics.
• Proven track record building a product from zero to serving real traffic.
• Self‑starter who can own an area with minimal direction and adapt to adjacent tasks as needed.

Required skills

• Kubernetes
• NVIDIA Dynamo
• Modern inference frameworks and serving engines
• Model deployment and serving
• Monitoring, gateways, and endpoint setup for production services
• KV‑cache orchestration
• Autoscaling and performance optimization
• Quantization, batching strategies, kernel‑level tuning (optional)
• RDMA and high‑performance networking (optional)