LLM Inference & GPU Systems Consultant

About the role

We are looking for an experienced AI Infrastructure Runtime Engineer to design, deploy, and operate a large‑scale on‑prem LLM inference platform. The environment runs on NVIDIA H200 GPU clusters within an OpenShift AI ecosystem, serving self‑hosted open‑source models such as Llama. This position focuses exclusively on inference workloads and does not involve model training or fine‑tuning.

Key responsibilities

• Optimize NVIDIA GPU runtime for token generation, including prefill/decode and KV‑cache management.
• Deploy and maintain inference engines such as vLLM and TensorRT‑LLM.
• Maximize GPU throughput through batching, latency tuning, and workload orchestration with RunAI and Kubernetes.
• Manage the full Hugging Face model lifecycle: onboarding, deployment, monitoring, and retirement.
• Operate and support the OpenShift AI container platform for GenAI workloads.

Required profile

• 8+ years of experience as an LLM Systems Engineer or AI Infrastructure Runtime Engineer.
• Extensive hands‑on work with NVIDIA H200 clusters and runtime optimization techniques.
• Proven expertise in OpenShift AI and GPU orchestration tools like RunAI.
• Demonstrated success managing Hugging Face model deployments.

Required skills

• NVIDIA H200 GPU clusters
• GPU runtime optimization (prefill/decode, KV‑cache)
• vLLM inference framework
• TensorRT‑LLM
• RunAI workload orchestration
• Kubernetes GPU scheduling
• OpenShift AI platform
• Hugging Face model lifecycle management